[ I am initially copying only a few lists since they seem like the most impacted projects and I didn't want to spam all the mailing lists. Please let me know if other lists should be included. ]
I submitted D68230 for review but this is not about that patch per se. The patch allows update_cc_test_checks.py to process tests that should check target asm rather than LLVM IR. We use this facility downstream for our end-to-end tests. It strikes me that it might be useful for upstream to do similar end-to-end testing. Now that the monorepo is about to become the canonical source of truth, we have an opportunity for convenient end-to-end testing that we didn't easily have before with svn (yes, it could be done but in an ugly way). AFAIK the only upstream end-to-end testing we have is in test-suite and many of those codes are very large and/or unfocused tests. With the monorepo we have a place to put lit-style tests that exercise multiple subprojects, for example tests that ensure the entire clang compilation pipeline executes correctly. We could, for example, create a top-level "test" directory and put end-to-end tests there. Some of the things that could be tested include: - Pipeline execution (debug-pass=Executions) - Optimization warnings/messages - Specific asm code sequences out of clang (e.g. ensure certain loops are vectorized) - Pragma effects (e.g. ensure loop optimizations are honored) - Complete end-to-end PGO (generate a profile and re-compile) - GPU/accelerator offloading - Debuggability of clang-generated code Each of these things is tested to some degree within their own subprojects, but AFAIK there are currently no dedicated tests ensuring such things work through the entire clang pipeline flow and with other tools that make use of the results (debuggers, etc.). It is relatively easy to break the pipeline while the individual subproject tests continue to pass. I realize that some folks prefer to work on only a portion of the monorepo (for example, they just hack on LLVM). I am not sure how to address those developers WRT end-to-end testing. On the one hand, requiring them to run end-to-end testing means they will have to at least check out and build the monorepo. On the other hand, it seems less than ideal to have people developing core infrastructure and not running tests. I don't yet have a formal proposal but wanted to put this out to spur discussion and gather feedback and ideas. Thank you for your interest and participation! -David _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev