On Fri, May 11, 2012 at 2:58 PM, Roberto Galoppini <[email protected]> wrote: > On Fri, May 11, 2012 at 5:06 AM, Rob Weir <[email protected]> wrote: > >> SourceForge has a nice REST API to query for download stats and return >> them in JSON objections. Unfortunately, our directory structure for >> AOO 3.4 is rather odd, with English downloads in one place, >> translations in another directory, and mixing hashes, installs and >> languages packs altogether. So getting these stats is a little >> painful. You can't just get the numbers of a single directory and be >> done. It is more complicated than that. >> >> Also, the SF API seems to be rate limited, or at least I'm getting >> errors if I query it too much. That's understandable. >> > > Rob, can provide me with more info about this, so that we can investigate > it further?
Hi Roberto, At the python level the error is on a urllib.urlopen().read(), with an returned error of: "IOError: [Errno socket error] [Errno 10054] An existing connection was forcibly closed by the remote host" I find that this happens when I make many (> 50) requests in a short period of time (1 or 2 minutes). My solution right now is to maintain a local disk cache of prior returned results. Not only does this reduce the number of requests I send SF, but it also improves the speed of the report generation. In any case, I'm happy with the caching solution, so this is not a blocking issue for me right now. -Rob > We plan to share some stats figures next week, as we did previously for > Extensions and Templates. > > > Thanks, > > Roberto > > >> >> So.... I'm coding a simple download stats app, in python, that will >> collect together all the relevant stats and produce reports. It >> caches on disk JSON objects that have already been retrieved, which >> eliminates the throttling issues as well as greatly improves >> performance. >> >> Not quite done, but I'll check it in (where?) when it is fully >> debugged and validated. My goal is to have solid numbers for the one >> week mark next Tuesday. And from what I'm seeing so far, the numbers >> will be amazing. >> >> But two quick questions to help me finish this: >> >> 1) Historically, what did OOo report as "downloads"? Was this just a >> count of full installs? Or language packs as well? >> >> 2) It is easy to produce downloads by language and platform, since our >> installs are already defined that way. But I can also report >> per-country. Is that interesting to anyone? For example, in Canada, >> the most popular downloads are X, Y, Z. >> >> >> -Rob >> > > -- > ==== > This e- mail message is intended only for the named recipient(s) above. It > may contain confidential and privileged information. If you are not the > intended recipient you are hereby notified that any dissemination, > distribution or copying of this e-mail and any attachment(s) is strictly > prohibited. If you have received this e-mail in error, please immediately > notify the sender by replying to this e-mail and delete the message and any > attachment(s) from your system. Thank you. >
