[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)
https://github.com/mtrofin approved this pull request. Thanks for adding license info! https://github.com/llvm/llvm-project/pull/72319 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)
boomanaiden154 wrote: I believe this is ready to go now. All the tests have been converted to lit-style tests and everything has been wired up into a `check-mlgo-utils` CMake target. I talked with Mircea about the absl dependency. It's not in the tests anymore and the plan is to refactor in a subsequent commit the rest of the utilities that use absl to the generic python versions. For now the tests have been gated to actually having `absl` installed. https://github.com/llvm/llvm-project/pull/72319 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)
mtrofin wrote: > > Would it be also possible to remove the dependency on > > [Abseil](https://github.com/abseil/abseil-py)? None of the existing scripts > > in LLVM use it and I don't think we should be introducing this dependency. > > It looks like Abseil is only used for flag parsing, logging and testing; > > those should be straightforward to replace with standard libraries like > > `argparse`, `logging` or `unittest`. > > Yes. My plan was to remove the dependency on abseil as well. My plan was to > get this landed with all the infrastructure setup and the code basically just > directly copied and then remove the abseil dependency in a follow-up patch so > that the different pieces get reviewed appropriately. Ah, if you can drop the abseil dependency, the dependency problem for tests goes away. May be worth doing the abseil dropping bit in this patch, too? https://github.com/llvm/llvm-project/pull/72319 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)
@@ -0,0 +1,6 @@ +# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. mtrofin wrote: Ack - thanks for clarifying the directory structure; looking around, there doesn't seem to be much of a precedent either; so... how about `llvm/utils/mlgo-utils/mlgo`? (tests moving to lit sounds perfect) https://github.com/llvm/llvm-project/pull/72319 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)
@@ -0,0 +1,6 @@ +# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. mtrofin wrote: why do we need `mlgo/mlgo` - could we just do `mlgo/corpus` and `mlgo/tests/corpus` - this leaves it open to add later non-corpus related things if needed. Shouldn't the tests go under `llvm/test/tools` like the other utils? There should be a way to disable them if python version is under 3.8. Also the `lit.local.cfg` should probably go through the dependency list of the package and skip the tests if the packages aren't available. I assume we can do that off `pyproject.toml` https://github.com/llvm/llvm-project/pull/72319 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)
@@ -0,0 +1,12 @@ +# MLGO Python Library + +This folder contains the MLGO python library. This library consists of telling mtrofin wrote: I wouldn't call it "the" MLGO python library, rather it's utilities for MLGO. Also please read the rest of the para (esp. first line) https://github.com/llvm/llvm-project/pull/72319 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)
https://github.com/mtrofin commented: I'm assuming the files were drop-in copied from google/ml-compiler-opt, so no need to comment there; also that we would subsequently delete them from there and depend on this package. https://github.com/llvm/llvm-project/pull/72319 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
