[
https://issues.apache.org/jira/browse/AIRAVATA-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379921#comment-17379921
]
ASF subversion and git services commented on AIRAVATA-3322:
-----------------------------------------------------------
Commit 2137d83951bf1e0e87936fb3c03931adcba118a8 in airavata-django-portal's
branch refs/heads/develop from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=2137d83 ]
AIRAVATA-3322 Pull the airavata-python-sdk from PyPI
> Add paging to experiment statistics
> -----------------------------------
>
> Key: AIRAVATA-3322
> URL: https://issues.apache.org/jira/browse/AIRAVATA-3322
> Project: Airavata
> Issue Type: New Feature
> Components: Django Portal, Registry API
> Reporter: Marcus Christie
> Assignee: Marcus Christie
> Priority: Major
>
> The getExperimentStatistics API method works well for small numbers of
> experiments but doesn't scale well to thousands of experiments, which some
> popular gateways like [seagrid.org|https://seagrid.org] seeing this level of
> activity. For example, for SEAGrid we had to up the Apache/Django timeout to
> 5 minutes to load statistics for a few months for one compute resource, and
> even then it didn't always load.
> I added slow query logging in the MariaDB but that turned up nothing so I
> don't believe the queries need to be tuned. I think the main culprit is that
> thousands of ExperimentSummary instances are getting populated and returned
> by the method. It would be better to page in experiments and only return the
> first, say, 50 experiments in each status category.
> Currently the ExperimentSummaries are loaded for each ExperimentState and
> they are counted. Instead, we'd need two queries per state, one to load the
> count as a scalar and another to load a page's worth of ExperimentSummaries.
> To preserve backwards compatibility, it might be best to just make a new
> method and then in time phase out the old one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
