It is wonderful to see that there's a team of folks with enough time and tuits[1] to offer substantial contributions in the domain of CPAN testing. It's been so long that I had stopped asking, and hoping, for someone to join me on the main project. Now, I will hope that we can all find a way to work together on one project instead of continuing to work separately on multiple projects.
--- The Magpie project sounds like my original plan for CPAN Testers when I took over from Barbie back in 2016: Just swap in "Mojolicious" for "Dancer2" + related frontend tech, and the roadmap is the same. And I have substantially achieved it: The frontend is Mojolicious and provides data in a compatible way for the Matrix, Andk's analysis site, and some other client libraries on CPAN for getting test report data. The backend is Minion and DBIx::Class, which are current and accepted as good practice by the Perl community. It accepts reports in the old Metabase format and a new, more-detailed format[2]. To go along with this, I set up a modern production infrastructure with telemetry, monitoring, and alerting. In fact, Magpie is so similar to my goals that I'm not sure the bus factor is improved by having two reporting platforms. The current living code, minus the parts of the CGI-based app that are still running for compatibility, _is_ a modern rewrite that is backwards compatible and able to use Postgres[3]. The reason things with CPAN Testers are currently as they are has two parts: On one side there are the vagaries of a 100%-volunteer, donation-supported project[4]. On the other side are the substantial technical requirements of the reporting system itself. Finding people to join the team has been the most difficult part of this project, despite my marketing efforts. There are some folks who can log in to the CPAN Testers servers to perform maintenance, though that list is small: I gave access to anyone who asked and demonstrated a need. I would give access to anyone developing on the reporting platform, but nothing panned out and I stopped asking. My efforts at improving the developer experience by building a quick-start dev environment[5] have not yet helped. But it's little surprise that motivated volunteers are scarce: The CPAN Testers database is nearing 1 terabyte of data, and the web application serves over 10 requests per second. When combined with a budget of $0 US, these requirements squash all plans for ease, cleanliness, and purity of implementation. The former app solved these problems by generating static files to serve quickly. My current rewrite solves it by having more and better hardware, upgrading the database software, and improving performance of the database schema. It isn't enough: There are still problems, and solving those problems is hard work that requires sustained, dedicated effort by developers with experience doing so. --- So that's where the project currently stands. I put out the big fires when my monitors tell me to or I am otherwise notified of a significant outage, as that's what I can handle. I work on improvements when I have dedicated time like the Perl Toolchain Summit, or other conference time, or as a "vacation" from my day job at GSG. Offers to help have been shallow and infrequent, so I avoid wasting effort and demoralizing myself with onboarding people who then disappear. Otherwise, I try to salve my anxiety over the precarious state of CPAN Testers by thinking about how to unravel the Gordian knot[6] that is the CPAN Testers database[7]. And, I take comfort in the knowledge that if things ever get truly and irrevocably FUBAR[8], the problem of the data size goes away and the new most-difficult problem becomes getting over that loss having been caused by my own failure and neglect. Hence the honest, but possibly overstated, message on the CPAN Testers main page: One of the Bytemark web servers was shut down for non-payment[9]. That server hosted the legacy app, which was (at the time) still the primary web UI. I was alerted to the outage by my monitoring, and I switched the site over to the prototype web UI I'd been working on since 2016. Turns out, lucky me[10], it mostly worked. And then the Bytemark server came back to continue doing the one main thing the prototype was missing (the e-mail reporting). Even though the server I lost came back weeks later, I left the prototype online and the "degraded" message up: It felt good to have the thing I'd been working on for almost a decade finally see the light of day. Up until that point, everything I improved was on the backend and invisible, toiling in obscurity, except for little prototypes I slapped together to demonstrate what could be done with minimal effort to try to entice a volunteer to join the project. And I thought, "Who knows, maybe the dramatic change and highly-visible request for help will generate interest." --- And now, as announced on this e-mail thread, there's a team of folks who are ready, willing, and able[11] to help with what needs doing. Again, I don't think it's necessary to start from scratch and have another project go through a lengthy and disruptive maturing process to handle the necessary scale, re-build all the compatibility layers, and maintain a second production server stack. I think it would be much more effective to continue to evolve the current application, which is less than 8 years old and using up-to-date software development practices (both for Perl and in general). If collaboration is not possible, I have no interest in competition: Competition does not build community, which conflicts with both my goals for this project and my personal beliefs, and two projects that fill the same role can only be in competition for developer resources and community adoption. Instead, I can work with the Magpie team on an orderly transition[12] of the CPAN Testers domain names[13], servers[14], and any relevant accounts (like the Fastly account), and be accessible to lend any knowledge and expertise I have gained from my tenure. Doug Bell d...@preaction.me [1]: A joke about doing something when one gets "a round tuit" (around to it), someone (I forget) at a previous toolchain summit brought small wooden tokens labelled "tuits". I'm not sure where mine went, which might be part of why I've done less open source work recently ;) [2]: This format was developed at the Perl Toolchain Summit in Oslo (2018) and it is, as yet, unused by any reporting client. The new format would enable some better reporting features to be created, but there's a problem of getting folks reporting with the new format, and/or analyzing the current format to fill in details in the new format. [3]: Indeed, all that I feel it is missing are: A native e-mail report sender, as right now it uses the old processes, and a user login and configuration portal for that e-mail report sender, based on a PAUSE OAuth2 gateway (a discussed but defunct project). [4]: These days, outside of the donated hardware and rack space from Summit and Bytemark, the money to cover other hardware and expenses is donated by me alone, which I am happy to do and stopped futilely begging for help with years ago. [5]: https://github.com/cpan-testers/cpantesters-deploy?tab=readme-ov-file#getting-started-development-environment [6]: https://en.wikipedia.org/wiki/Gordian_Knot - Alexander the Great famously (possibly apocryphally) cut the knot with his sword after being told it could not be untied by scholars and had become a legend [7]: My current best plan for moving forward is a hybrid online/offline data approach with some old data compressed in static files, more-recent data in the database, and have the CPAN::Testers::Schema module present a unified API to both the backend processing and frontend web applications. [8]: Fucked Up Beyond All Repair/Recovery/Recognition [9]: By whom, I haven't the foggiest. Barbie? Mark Keating of the Enlightened Perl group? Neil Bowers who has previously acted as an intermediary for me with Bytemark? Or are they just donated by Bytemark? If I knew, I might've been in the loop that it was in danger of being shut down... [10]: Past me didn't let present me down for once: The attention to detail and forward-thinking he displayed paid off. [11]: An English idiom used to describe someone who is capable of and eager to do something. Used often in legal contexts. [12]: Preferably to an organization instead of an individual, as I've been trying to do since 2016. [13]: These are currently held in Cloudflare under, as I understand, The Perl Foundation's umbrella account. [14]: The current Summit database and Bytemark web servers, precarious as they may be, and the ones I pay for hosted by NetActuate (formerly RootBSD). > On Mar 28, 2025, at 9:03 AM, Ruth Holloway <r...@hiruthie.me> wrote: > > I. Rationale > > Currently, the CPANTesters.org <http://cpantesters.org/> site has been > running "in a degraded state" for some time; many reports that were > previously available simply aren't there any more, and the platform has > frequent outages in which it will not accept new reports, as well as a > non-functioning API for requests. Doug Bell (preaction) has been the primary > maintainer of this crucial piece of infrastructure for some time, but has > lately been hard to reach for questions, concerns, and even offers of > assistance. > > This creates a "Bus Factor" problem for the larger Perl community; both > module authors and users, as well as the core developers, depend on CPAN > testing for assurance that their work is sound and functioning as designed > across the many platforms and versions of Perl that exist in the wild. > > While I (and others) will be attending the Toolchain Summit with the specific > goal of working with Doug to help make the existing infrastructure more > robust and capable, I believe there is also room in our community for > parallel development of a new system, which is being called the "Perl Magpie." > > The existing code contains artifacts of testing dating back to the end of the > 20th century; the Magpie team believes that it is time for an end-to-end > redesign, that preserves the functionality of the 25-year-old code, while > opening up prospects for new ways of reporting tests. > > II. Design philosophy and program structure > > Magpies, birds of the family Corvidae, are seen throughout Eurasia, in a > variety of species. They are known to collect found objects, and will > reputedly display found objects to peers and other animals that they may > interact with. They are regarded by scientists to be very intelligent, > including the ability to make and use tools, imitate human speech, and work > in teams. > > Inspired by the magpie, we wish to develop a simply-structured web > application that can collect CPAN testing and other data about CPAN modules, > and present it in multiple friendly formats upon request. Obviously, we need > backwards-compatibility in such an application, so that existing testers may > send it test results with only minor changes to their configuration. > Additionally, we would like the application to provide additional means of > ingesting tests, and a wide variety of API accessors to read back the > information after ingestion, so that others may generate creative new ways of > displaying the data. > > In my daily work, I write a lot of Dancer2 applications, using PostgreSQL as > a database, with simple JavaScript/jQuery front-ends. Dancer2 handles the web > routing engine, and with plugins, can provide for > authentication/registration, RSS feeds, and a fully RESTful API. Using > JavaScript and jQuery lets us send some of the rendering work to the browser, > offloading that work from the server and allowing for customized rendering in > browsers focused on accessibility for users with (particularly) visual > disabilities. > > Our first-phase design philosophy centers on the KISS principle--make it as > complex as necessary, and no moreso, and not adding features or functionality > because "we might want that someday," or prejudging the use cases, instead > waiting for them to arise. We begin by ensuring that existing functionality > is preserved--to a reporter using Test::Reporter::Transport::Metabase, it > should "look like" a Metabase, responding to the same URLs that the existing > system will do. Thus, a test reporter could change the address (not the whole > URL) in their .cpanreporter configuration file, and submit tests--it should > Just Work, and properly respond to a correctly-formatted test report by > ingesting it, and promptly making it available for reporting. > > The database structure is simple, with the "Test" table storing a compressed > text of the report, and extracting key fields for aggregate reporting needs > to indexed fields (Perl version, distribution, author, tester, OS name, OS > version, platform/config options, and the test result). As we coax out new > information for the Magpie to collect, the DB schema will expand > incrementally. As much as possible, we'll avoid "statistics" tables for the > aggregation of data, until and unless we reach a point that the database > engine can't handle the load for us. While real-world rate data for this > dataset is unavailable to us at this time, I have worked with and developed > applications that are handling multiple-millions of records, summing > numerical data with sub-one-second times. > > Layout is largely left to the remote user of the data, using simple > JavaScript for rendering. Simple templates are created using > Template::Toolkit to provide a default interface, and APIs will be created > and documented for consumption in ways we haven't conceived. > > III. Feature Roadmap, and current status of work > > The repository for the code is at https://github.com/Perl-Magpie/Perl-Magpie > <https://github.com/Perl-Magpie/Perl-Magpie>, and pull requests are, of > course, welcome! > > V1.0: The basics -- Hope to have this completed and deployed prior to PTS May > 1: > * Accept a properly-formatted test result from cpanm-reporter, using the > Metabase transport pointing at its address. (DONE) > * Display a matrix for a given distribution, showing all available test > results in an informative aggregate page. (DONE) > * Allow for a browser click on the matrix elements, to drill down to a list > of tests that meet the desired criteria combination (OS, Perl Version, > Distribution, in any combination. (IN PROGRESS) > * Allow for a report ID to be clicked on, as in the current matrix system, > to allow display of a specific test report. > * Create a front-end page to allow for searching on module or distribution > name, which if successful, will return the matrix for the most-recent release > of a module, or a specific version, if any tests are available. > * Provide for appropriate responses in the case of malformed test reports > to the "Metabase" transport route, and for "No results available" in the case > of a search that does not hit in the database. > * Add robust external monitoring of services to ensure service reliability > and uptime for all users of the Perl ecosystem. This would include: API > health, Ping thresholds, disk usage, CPU load, backend health, etc which > would be published to an external site for community visibility. > * Optionally have health results published to IRC or the CPT mailing list > with information about the platform and its responsiveness for community > consumption. This should help prevent the "bus factor" issue we have today, > and allow other people with supporting skillsets to get involved. > > V1.1: Make it play with the rest of the ecosystem > * Utilizing a cron job or other strategy, capture the current Metabase > "log.txt" periodically, and use that to accumulate test results currently > coming in to the legacy Metabase. > * Provide for a "log.txt" result search for the Magpie, similarly > formatted, so testers can see that their tests are being ingested properly. > * Create an API that could be used by MetaCPAN and others to provide a fast > summary of results in JSON format for a given distribution. > * Work out some sort of mechanism to associate one or more Metabase UUIDs > to names and email addresses, to replicate the old "registration/moderation" > behavior after CPAN::Reporter configuration commands. > * Add some top-lists and interesting data to the front page--top testers, > most-tested, most-failed, etc. > * Add email functionality to allow for emailing failed tests to module > developers (including opt-out and summary capability). > * Create RSS feeds for distributions and CPAN authors, to give test reports > via RSS. > * Add "other versions" to the matrix page, to allow for checking tests on > prior/other versions of a distribution. > > V1.x: Ensure that other CPAN test tooling besides cpanm-reporter is > functional. > > V2.0: "Out there. Thataway." > * Possibilities without end--a new reporter module for CPAN that sends > reports to the Magpie via a new API? Create and/or collect CPANTS > information? Ideas are welcome, and it's time to think outside of the box.
