[
https://issues.apache.org/jira/browse/AIRAVATA-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Christie resolved AIRAVATA-3322.
---------------------------------------
Resolution: Fixed
> Add paging to experiment statistics
> -----------------------------------
>
> Key: AIRAVATA-3322
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3322
> Project: Airavata
> Issue Type: New Feature
> Components: Django Portal, Registry API
> Reporter: Marcus Christie
> Assignee: Marcus Christie
> Priority: Major
>
> The getExperimentStatistics API method works well for small numbers of
> experiments but doesn't scale well to thousands of experiments, which some
> popular gateways like [seagrid.org|https://seagrid.org] seeing this level of
> activity. For example, for SEAGrid we had to up the Apache/Django timeout to
> 5 minutes to load statistics for a few months for one compute resource, and
> even then it didn't always load.
> I added slow query logging in the MariaDB but that turned up nothing so I
> don't believe the queries need to be tuned. I think the main culprit is that
> thousands of ExperimentSummary instances are getting populated and returned
> by the method. It would be better to page in experiments and only return the
> first, say, 50 experiments in each status category.
> Currently the ExperimentSummaries are loaded for each ExperimentState and
> they are counted. Instead, we'd need two queries per state, one to load the
> count as a scalar and another to load a page's worth of ExperimentSummaries.
> To preserve backwards compatibility, it might be best to just make a new
> method and then in time phase out the old one.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
