On 2018-02-13 at 21:40, James Andrewartha wrote:

> On 14/02/18 10:25, Paul Kosinski wrote:
> 
>> Does anyone agree with me that such a "scorched earth" policy is 
>> unreasonable?
> 
> Absolutely. If Mozilla doesn't change their policy, I'll see if we
> can archive all of addons.mozilla.org for the Internet Archive.

I believe I'm one of the people Jorge mentioned as having been pointed
to API documentation, et cetera, about how to pull addons from AMO en
masse for a local archive.

The pointed-to API documentation was sufficiently far from user-friendly
that I couldn't do much with it when starting from zero. I wound up
finding someone else's existing Python scripts, updating them to
actually work (for which the API documentation was helpful), and using
them to pull down (what should be) the entire repository of XPIs -
including all historical versions - and what appears to be related
metadata in JSON form.

The result does not appear to be remotely suited for putting up an
AMO-like site, or even a readily accessible archive site (much less a
usable AMO mirror), without a lot of work that I'm not remotely prepared
to do; I looked for a repository of the code that runs AMO, but did not
find one. Still, I do have that apparently-full archive of
publicly-downloadable addon versions, as of sometime in early August of
2017. For reference, it comes to ~21GB.

https://github.com/andymckay/addons-server-mirror.git is the repo where
I found the script I tweaked. It didn't work as downloaded, but I don't
know whether that's because it never had or because something external
had changed in the meantime. The tweaks aren't necessarily suitable for
general usage (IIRC, one of the things they do makes the script more or
less incapable of doing anything but exactly what I needed it to), but I
can provide them if needed.

I plan to update (if not completely re-download) this local archive as
close to the last minute before AMO purges classic addons as I can.
Unfortunately I'm not remotely in a position to be able to make the
archive itself publicly accessible, but if someone who is wants to do it
in my place, I could probably provide what I already have.

That said, any further Mozilla-list discussion of this should probably
go on dev-addons. I'm already subscribed over there, so I'll catch any
conversations that do get started; I'm replying here this time only so
that interested parties to this thread who aren't subscribed to
dev-addons don't miss the reply.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Enterprise mailing list
Enterprise@mozilla.org
https://mail.mozilla.org/listinfo/enterprise

To unsubscribe from this list, please visit 
https://mail.mozilla.org/listinfo/enterprise or send an email to 
enterprise-requ...@mozilla.org with a subject of "unsubscribe"

Reply via email to