On Mon, Sep 13, 2021 at 12:49:21PM -0600, Tim Flink wrote:
It turns out that the configuration for rhbz was changed without an
announcement and now the max number of bugs returned for any query is 20. Some
of the queries that we use in blockerbugs return more than 20 bugs so I have a
hotfix to deal with the new pagination requirements. I've attached the patch to
this email.
I'd like to apply this patch to production. Unfortunately, we're in the middle
of a non-trivial rewrite that's currently on stg so testing the patch isn't
really an option and that code isn't ready for production use right now.
Thanks,
Tim
diff --git a/blockerbugs/util/bz_interface.py b/blockerbugs/util/bz_interface.py
index 471140f..a9f90d7 100644
--- a/blockerbugs/util/bz_interface.py
+++ b/blockerbugs/util/bz_interface.py
@@ -29,11 +29,14 @@ from xmlrpc.client import Fault
from blockerbugs import app
+# rhbz has been updated to have a max of 20 results returned
+BUGZILLA_QUERY_LIMIT = 20
If I understood correctly, you can request more than 20 results, but if you
don't specify how many you want, you'll only get 20.
I believe if you set it to 0 you may get everything, but this needs to be
confirmed empirically.
Anyway, I would check first if you can't retrieve 100 or 200 results in one go
instead of doing iterations of 20 ;-)
base_query = {'o1': 'anywords',
'f1': 'blocked',
'query_format': 'advanced',
- 'extra_fields': ['flags']}
-
+ 'extra_fields': ['flags'],
+ 'limit': BUGZILLA_QUERY_LIMIT}
class BZInterfaceError(Exception):
"""A custom wrapper for XMLRPC errors from Bugzilla"""
@@ -77,7 +80,8 @@ class BlockerBugs():
#
https://bugzilla.stage.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED
#
&bug_status=POST&bug_status=MODIFIED&classification=Fedora&component=anaconda&f1=component
#
&o1=changedafter&product=Fedora&query_format=advanced&v1=2013-03-21%2012%3A25&version=19
- def get_bz_query(self, tracker: int, last_update: datetime.datetime = None)
-> dict[str, Any]:
+ def get_bz_query(self, tracker: int, last_update: datetime.datetime =
None, offset: int = 0
+ ) -> dict[str, Any]:
If you can set the limit to 0 to retrieve everything, you may want to default to
None rather than 0.
"""Build a Bugzilla query to retrieve all necessary info about all
bugs which block the
`tracker` bug.
@@ -129,6 +133,9 @@ class BlockerBugs():
'f10': 'CP'
})
+ if offset > 0:
And if you default to None, this will need to be tweaked.
+ query.update({'offset': offset})
+
return query
def query_tracker(self, tracker: int, last_update: Optional[datetime.datetime] = None
@@ -139,8 +146,21 @@ class BlockerBugs():
:param last_update: If provided, the query is modified to ask only
about bugs which have
recent modifications; otherwise asks about all
bugs.
"""
- query = self.get_bz_query(tracker, last_update)
- buglist = self.bz.query(query)
+
+ buglist = []
+ last_query_len = BUGZILLA_QUERY_LIMIT
+
+
+ # this is a hotfix hack to work around the sudden config change in
rhbz where the max
+ # number of bugs returned for a query is 20
+ # it seems to be working for now but may need more work going forward
+ while last_query_len == BUGZILLA_QUERY_LIMIT:
+
+ new_query = self.get_bz_query(tracker, last_update,
offset=len(buglist))
+ new_buglist = self.bz.query(new_query)
+ buglist.extend(new_buglist)
+ last_query_len = len(new_buglist)
+
Looks alright to me otherwise.
Pierre
_______________________________________________
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
