RKuttruff opened a new pull request, #275: URL: https://github.com/apache/incubator-sdap-nexus/pull/275
Pagination updates have led to an inefficient querying of the whole `doms_data` table for a given execution id when just trying to fetch a limited page of primary matches. When in cases such as `matchOnce=false`, where there can be significantly more secondaries than primaries, this wastes an incredible amount of time. Now the secondary fetch consists of queries targeted to the primary ids in the page. While this does get a speedup (benchmark test 27m -> 6m for execution of 661 primaries and 2,116,205 secondaries with 50 primaries per page), it does require a modification to the `doms_data` primary key. This will, unfortunately, necessitate either dropping the table entirely and losing all retained executions, or using the included script `tools/doms-data-tools/update_doms_data_pk.py` to copy the data to a temp table and back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
