On Mon, Sep 13, 2021 at 12:49:21PM -0600, Tim Flink wrote:
> It turns out that the configuration for rhbz was changed without an
> announcement and now the max number of bugs returned for any query is 20.
> Some of the queries that we use in blockerbugs return more than 20 bugs so I
> have a hotfix to deal with the new pagination requirements. I've attached the
> patch to this email.
>
> I'd like to apply this patch to production. Unfortunately, we're in the
> middle of a non-trivial rewrite that's currently on stg so testing the patch
> isn't really an option and that code isn't ready for production use right now.
>
> Thanks,
>
> Tim
> diff --git a/blockerbugs/util/bz_interface.py
> b/blockerbugs/util/bz_interface.py
> index 471140f..a9f90d7 100644
> --- a/blockerbugs/util/bz_interface.py
> +++ b/blockerbugs/util/bz_interface.py
> @@ -29,11 +29,14 @@ from xmlrpc.client import Fault
>
> from blockerbugs import app
>
> +# rhbz has been updated to have a max of 20 results returned
> +BUGZILLA_QUERY_LIMIT = 20
If I understood correctly, you can request more than 20 results, but if you
don't specify how many you want, you'll only get 20.
I believe if you set it to 0 you may get everything, but this needs to be
confirmed empirically.
Anyway, I would check first if you can't retrieve 100 or 200 results in one go
instead of doing iterations of 20 ;-)
> base_query = {'o1': 'anywords',
> 'f1': 'blocked',
> 'query_format': 'advanced',
> - 'extra_fields': ['flags']}
> -
> + 'extra_fields': ['flags'],
> + 'limit': BUGZILLA_QUERY_LIMIT}
>
> class BZInterfaceError(Exception):
> """A custom wrapper for XMLRPC errors from Bugzilla"""
> @@ -77,7 +80,8 @@ class BlockerBugs():
> #
> https://bugzilla.stage.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED
> #
> &bug_status=POST&bug_status=MODIFIED&classification=Fedora&component=anaconda&f1=component
> #
> &o1=changedafter&product=Fedora&query_format=advanced&v1=2013-03-21%2012%3A25&version=19
> - def get_bz_query(self, tracker: int, last_update: datetime.datetime =
> None) -> dict[str, Any]:
> + def get_bz_query(self, tracker: int, last_update: datetime.datetime =
> None, offset: int = 0
> + ) -> dict[str, Any]:
If you can set the limit to 0 to retrieve everything, you may want to default to
None rather than 0.
> """Build a Bugzilla query to retrieve all necessary info about all
> bugs which block the
> `tracker` bug.
>
> @@ -129,6 +133,9 @@ class BlockerBugs():
> 'f10': 'CP'
> })
>
> + if offset > 0:
And if you default to None, this will need to be tweaked.
> + query.update({'offset': offset})
> +
> return query
>
> def query_tracker(self, tracker: int, last_update:
> Optional[datetime.datetime] = None
> @@ -139,8 +146,21 @@ class BlockerBugs():
> :param last_update: If provided, the query is modified to ask only
> about bugs which have
> recent modifications; otherwise asks about all
> bugs.
> """
> - query = self.get_bz_query(tracker, last_update)
> - buglist = self.bz.query(query)
> +
> + buglist = []
> + last_query_len = BUGZILLA_QUERY_LIMIT
> +
> +
> + # this is a hotfix hack to work around the sudden config change in
> rhbz where the max
> + # number of bugs returned for a query is 20
> + # it seems to be working for now but may need more work going forward
> + while last_query_len == BUGZILLA_QUERY_LIMIT:
> +
> + new_query = self.get_bz_query(tracker, last_update,
> offset=len(buglist))
> + new_buglist = self.bz.query(new_query)
> + buglist.extend(new_buglist)
> + last_query_len = len(new_buglist)
> +
Looks alright to me otherwise.
Pierre
_______________________________________________
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
