I'll change my answer slightly: It seems like this error is happening from trying to download the resources.json file (hosted in the resources repo) too much. I suspect there are similar restrictions here. I think I can fix it too. I'll probably add a wait-and-retry for the resource downloader anyway as it'll make the downloader more stable.
-- Dr. Bobby R. Bruce Room 3050, Kemper Hall, UC Davis Davis, CA, 95616 web: https://www.bobbybruce.net On Tue, Mar 1, 2022 at 10:28 AM Bobby Bruce <bbr...@ucdavis.edu> wrote: > Nice find Gabe. That would certainly explain a lot. Turns out there's a > 50Gbps quota that will return a 429 error if hit: > https://cloud.google.com/storage/quotas. I'll see if I can get this > increased and, either way, I'll implement a wait-and-retry strategy in the > downloader for when this error is received. I suspect this is hit when lots > of Kokoro instances are spun up at one time. > > -- > Dr. Bobby R. Bruce > Room 3050, > Kemper Hall, UC Davis > Davis, > CA, 95616 > > web: https://www.bobbybruce.net > > > On Tue, Mar 1, 2022 at 2:08 AM Giacomo Travaglini via gem5-dev < > gem5-dev@gem5.org> wrote: > >> Hi Gabe, >> >> >> >> A possible workaround could be to use the --bin-path option in testlib >> >> >> >> Kind Regards >> >> >> >> Giacomo >> >> >> >> *From: *Gabe Black via gem5-dev <gem5-dev@gem5.org> >> *Date: *Tuesday, 1 March 2022 at 04:10 >> *To: *gem5 Developer List <gem5-dev@gem5.org> >> *Cc: *Gabe Black <gabe.bl...@gmail.com> >> *Subject: *[gem5-dev] resource downloader failure >> >> Hi folks. I've been trying to run tests locally, and I've been running >> into occasional flakiness due to a problem with the resource downloader. >> This reminds me somewhat of flakiness I was seeing in kokoro as well, so >> they are probably related. The error output is this: >> >> >> >> $ cat >> ./testing-results/SuiteUID:atomic-cpu_1-cores_classic_kvm-fork-run-test-GCN3_X86-x86_64-opt/TestUID:atomic-cpu_1-cores_classic_kvm-fork-run-test-GCN3_X86-x86_64-opt/simerr >> warn: <orphan X86Board>.kvm_vm already has parent not resetting parent. >> Note: kvm_vm is not a parameter of X86Board >> warn: (Previously declared as <orphan X86Board>.processor.kvm_vm >> HTTPError: HTTP Error 429: Too Many Requests >> >> At: >> /usr/lib/python3.10/urllib/request.py(643): http_error_default >> /usr/lib/python3.10/urllib/request.py(496): _call_chain >> /usr/lib/python3.10/urllib/request.py(563): error >> /usr/lib/python3.10/urllib/request.py(634): http_response >> /usr/lib/python3.10/urllib/request.py(525): open >> /usr/lib/python3.10/urllib/request.py(216): urlopen >> build/GCN3_X86/python/gem5/resources/downloader.py(77): >> _get_resources_json >> build/GCN3_X86/python/gem5/resources/downloader.py(207): >> get_resources_json_obj >> build/GCN3_X86/python/gem5/resources/resource.py(162): __init__ >> /home/gblack/gem5/work/tests/gem5/configs/boot_kvm_fork_run.py(205): >> <module> >> build/GCN3_X86/python/m5/main.py(434): main >> >> >> >> >> >> Is there a limit that needs to be bumped up? Some way to consolidate >> requests so we're not hitting the existing limit? >> >> >> >> Gabe >> IMPORTANT NOTICE: The contents of this email and any attachments are >> confidential and may also be privileged. If you are not the intended >> recipient, please notify the sender immediately and do not disclose the >> contents to any other person, use it for any purpose, or store or copy the >> information in any medium. Thank you. >> _______________________________________________ >> gem5-dev mailing list -- gem5-dev@gem5.org >> To unsubscribe send an email to gem5-dev-le...@gem5.org >> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > >
_______________________________________________ gem5-dev mailing list -- gem5-dev@gem5.org To unsubscribe send an email to gem5-dev-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
