[
https://issues.apache.org/jira/browse/COMDEV-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18131367#comment-18131367
]
Sebb commented on COMDEV-572:
-----------------------------
There was a problem with timeout retries, has now been fixed. Hopefully the
script will now be able to complete most days.
But the issue remains: the script makes many more requests than are necessary.
More data would need to be kept in order to support summaries such as the
rolling quarterly traffic, but generally storage is not an issue. Caching the
data would also speed up the job.
> Mailglomper keeps failing
> -------------------------
>
> Key: COMDEV-572
> URL: https://issues.apache.org/jira/browse/COMDEV-572
> Project: Community Development
> Issue Type: Bug
> Components: Reporter Tool
> Reporter: Sebb
> Priority: Major
>
> The 3rd party lists.a.o server was updated recently, and since then
> mailglomper2.py has been failing much more frequently, mostly with timeouts.
> The code has been updated to skip inactive lists, and to retry some errors
> once, but the errors still occur.
> I think the code needs a bit of a rethink to reduce the load on lists.a.o and
> make it more efficient.
> For example, at present the last 6 months stats are collected in parallel.
> This should mean they complete quicker, but the load on lists.a.o is
> increased.
> Also, why does the code need to fetch 6 months each time?
> Surely past months won’t change?
> There is also some scope for sharing mail data with the recent metrics code
> in projects.a.o
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
