Hello Lucas, On Sun, 03 Jul 2022, Lucas Nussbaum wrote: > Seeing that lintian got adopted, I got motivated into looking if I could > help on the lintian.d.o side, that is, provide up-to-date archive-wide > up to date to developers.
Thanks for working on this! > Since the architecture of lintian.d.o seemed quite complicated, I > instead decided to follow what worked for other UDD-based data importers > (such as the one that scans for new upstream versions). So my plan is > the following: > - use a UDD postgresql table for data storage > - use UDD to decide which packages need to be analyzed > - coordinate the analysis from UDD, but do the analysis itself on a > third-party 'worker' machine (since the process is quite CPU intensive) > - provide visualisation directly on https://udd.debian.org (similar > to https://udd.debian.org/dmd/ or https://udd.debian.org/bugs/) > - work with data consumers on how to best export the data from UDD to > them > > I know it feels a bit like NIH, but I believe the simpler design will > help in the long term... At least it helps to have something running in the short term. But there are really parts that I'd like to move to something more standardized that we can use for multiple tasks in the context of Debian. Paul already hinted at it but debusine is clearly meant to help with: - scheduling tasks - running tasks on multiple workers https://salsa.debian.org/freexian-team/debusine/ At this point, debusine is not doing much yet but I hope that we have laid out some good initial architecture to be able to share the workload on multiple workers. Currently the only "Task" that it knows how to run is "sbuild" and we don't have any data storage yet (i.e. the generated artifacts are not stored anywhere, it's up to the caller to pass some --post-build-commands to make something with the result). Storing data is the next milestone that we will work on. Maybe it's a bit early to try to use it for your use case, but if you want to give it a try, you are more than welcome to. Feel free to open tickets and ask questions too. Cheers, -- ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hert...@debian.org> ⣾⠁⢠⠒⠀⣿⡁ ⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/ ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS
