Re: [creduce-dev] Avoiding syntax warnings and errors
Hi John, > On 14 Mar 2018, at 14:53, John Regehr wrote: > > Regarding try-catch, yes, we'd like to have a pass for that. I added a note > about this to our TODO list, but mostly we're too busy for adding new stuff > these days. I quickly created a simple clang_delta pass to remove/simplify try-catch constructs. For more details have a look at the pull request: https://github.com/csmith-project/creduce/pull/157 For now it is fairly simple but I think it should help already against useless try-catch blocks. Later we could have a more advanced pass to keep only catch-blocks or to concatenate them with the try-block. Let me know what you think. Moritz
Re: [creduce-dev] cmake bug?
I'm currently on vacation but once I'm back next week I can have a look as well if it's still unsolved. Moritz > On 31 Jan 2018, at 05:39, John Regehr wrote: > > I have weak cmake skills but can try to help. > > In any case I just added this as an item on the TODO list so we don't forget, > in case we don't do it right away. > > John > > > On 01/30/2018 08:31 PM, Eric Eide wrote: >> John Regehr writes: >>> In souper we have similar foo.in -> foo transformations doing some >>> substitutions that do get run at "make" time. I don't know if there's >>> something special about C-Reduce that makes this unworkable. >> Now that you mention it, I vaguely remember trying `configure_file` and >> failing. I forget why I failed. Probably because I don't know what I'm >> doing. >> Maybe I should try again! >> I vaguely recall that one problem was that `configure_file` doesn't let one >> specify that the output should be executable. But I think I had problems >> with >> the actual substitutions, too? >> Eric.
Re: [creduce-dev] [RFC] Switching from Perl to Python
Hi Martin, I'm afraid that project slipped a bit. I was almost done with porting the existing functionality but then I had to focus on some other things. But if there is still interest in this project, also mainly from John and Eric, I could pick it up again. Currently I have again some more time that I could spend on finishing the rework in Python and also offer support at least during the transition period. Let me know what you think. Moritz > On 2 Jan 2018, at 13:48, Martin Liška wrote: > > Hello. > > I've working on GCC as a develop, creduce is my daily tool and I'm really > grateful we've got such a tool. > I'm also responsible for packaging of the project in openSUSE. And time to > time I see some warnings/errors > in the tool. To be honest, I don't like Perl much, so I always bail out any > debugging of the tool. > > I would be happy to see the tool re-written in Python and I'm curious about > any progress update > of the migration. From I've read in this discussion, the transition is very > close. > > Thank you, > Martin
Re: [creduce-dev] installing C-Reduce on windows
Hi Aya, The clang binaries you downloaded should contain everything that is needed (if I remember correctly). When installing clang, did you choose to add it to the PATH environment variable? If not you will add -DLLVM_DIR=install/dir/lib/cmake/llvm to your CMake commandline for C-Reduce. Please double check the path I typed it from memory. Regards, Moritz
Re: [creduce-dev] Making creduce more parallel
Hi, I still had no time to read everything in thia conversation but one thing about the line removal came to my mind. If I remember correctly the removals for one fixed granularity settings should not overlap. So merging should be easy be just picking one of the two files at random, searching for the part that has been remove in the other file (that need's some modification of the pass so that it reports what has been removed), and removing it from the first file as well. For the first runs where granularity is set to a high value we don't get a many parallel jobs but I guess that's ok because we potentially remove a large part at once. And later there should be no problem to get a lot of parallelism as long as the file is still larger enough. I guess after each merge we also would need to run an interestingness test, won't we? Regards, Moritz On 27 Jan 2017 18:27, John Regehr wrote: > > > How are the passes / reducers structured right now? If we can generate > > all of a pass's potential reductions up front, then they can be > > inserted into the queue in a random order to reduce the likelihood of > > conflicts. If the passes don't separate generating a potential > > reduction from testing it, then we may need to refactor more. > > Let's talk about the line reduction pass at granularity 1 (each variant > is created by removing one line). We're running it on a 1000-line file. > > The search tree here has 2^1000 leaves, so we certainly don't want to > try to generate all variants up-front. > > What we can do is speculate: assume the variants are unsuccessful > (statistically this is the right guess) and now we only have 1000 > variants, so that is feasible, but not particularly fast since we're > manipulating ~50 KB of text. Worse, this line of speculation becomes > more and more out of sync with the current best reduction, assuming that > some line removals succeed -- so merge conflicts are going to start > happening. > > The upshot is that while the coordinator can run ahead, it should run > only far enough ahead that the queue doesn't empty out. > > The API for a C-Reduce pass is purely functional. The pass takes a > current test case and a pass state object, and either produces a new > variant + pass state or else says that it has no more variants to > generate. This API is not designed to facilitate picking random variants. > > However, a quick hack to get a random variant is to just repeatedly > invoke the pass a random number of times. This is not fast but it'll > get things off the ground with very little effort. Some experimentation > will be required to determine how this parameter interacts with the > likelihood of merge conflicts. > > > I was imagining that the "orchestrator" process would spawn worker > > threads that spawn-and-wait on interestingness processes and use > > CSP-style channels to comminucate with the main thread that owns the > > queue. This would leverage the modularity of passes that > > generate potential reductions and of the interesting test. > > > > Something like this diagram: > > https://gist.githubusercontent.com/fitzgen/bf1acdc6dad217f2ed5accbabce9cf73/raw/981bff47be0f818e69041908eb63035fabb4e25a/orchestrator-diagram.txt > > > > > > I was planning on prototyping in Rust, which has channels in its > > standard library. Python's `queue.Queue` should also be able to handle > > the job. > > > > If you have other suggestions, I am all ears. > > This sounds fine, the only suggestion I have is that you might consider > using a network-friendly mechanism for the orchestration in case we > wanted to see how reduction across multiple machines works. Or at least > design things so that this isn't difficult if anyone wants to try it out > later. > > I would suggest that you start out dealing only with one or two passes, > perhaps the line remover and the token remover. These do some heavy > lifting, hopefully never crash, and are always easy to understand. > > John >
Re: [creduce-dev] Update to CMake-Based Build System
I have not tested cccp but it seems to be from the GNU family. I suspect that it will not work on Windows either. Instead I have found cppp (https://homes.cs.washington.edu/~mernst/software/cppp) which is a Perl script which should work on Windows. I will test later and come back with feedback. Regards, Moritz > On 28 Jun 2016, at 14:08, Eric Eide wrote: > > mor...@pflanzer.eu writes: > >> I can confirm that it works on OS X 10.11. > > Thanks! > >> Unfortunately it does not work natively on Windows because "unifdef" cannot >> be build with Visual Studio. Some dependencies are Posix only. I would >> suggest to make the build of "unifdef" conditional and to show a warning if >> its build is deactivated. The user could then build it by themselves using >> Cygwin etc. > > Thanks! I wonder if we could replace unifdef with a similar tool that does > build on Windows. (I recall cccp, but I don't know if it is > Windows-friendly.) > > Eric. > > -- > --- > Eric Eide . University of Utah School of > Computing > http://www.cs.utah.edu/~eeide/ . +1 (801) 585-5512 voice, +1 (801) 581-5843 > FAX
Re: [creduce-dev] cache
Hi, > I think it's a good idea. In terms of large memory consumption, perhaps we > could start caching when the size of the input test drops to some value (e.g. > 10k)? Or maybe an alternative would be to store the cached results as files on disk? Now that only every pass is cached it shouldn't be too much I hope. In this case a hash of the content could be used as key in the cache map, pointing to the location of the file (or rather the file name). Another thought I had but which might be wrong: Take the position of the pass within an iteration -- currently $pass_num in the Perl code --, the pass name and argument, and a hash of the file contents before the pass has executed and store this information in a cache. If we now come to the point -- in a later iteration -- where this information is already in the cache when we want to store it, can we then abort the reduction immediately because apparently nothing has changed during a whole iteration? Could that help to terminate the last round of the reduction when nothing changes earlier? Or have I forgot something that invalidates my logic? Regards, Moritz
Re: [creduce-dev] [RFC] Switching from Perl to Python
Hi John, It took me a good time longer than I expected but I now my Python version of C-Reduce is complete. I ported all passes to Python and except for the "skip key" feature all options are supported. The skip-key feature seems indeed to be quite a problem under Linux as there is no non-blocking read function which reads only a single character (not followed by ENTER). For now I stopped trying to implement the feature to see if there is further interest at all. I pushed the current version to the "python" branch of my repository at: https://github.com/mpflanzer/creduce/tree/python I added a second table to the spreadsheet for a speed comparison for the complete set of passes: https://docs.google.com/spreadsheets/d/1FIvuHr29X2T2H2wOrnGCU0BUM3NeQrvJY_GpKMVJRCA/edit?usp=sharing > Yes, absolutely, there's no requirement for a line-by-line rewrite. I'll have > to look at the code to find things I'm unhappy with, but the short version is > I've rewritten the C-Reduce core enough times that I'm pretty happy with it. I think I found a few minor bugs in some passes which I fixed in the Python version. So if the Python version has no future on its own I will compare it against the Perl version and report the issues I found. The only major change -- I think -- is that interestingness test are now Python modules, i.e. Python scripts. Currently each script has to define a "run" and a "check" function. Both function take a list of test cases as inout, "check" has to return True or False depending on the interestingness of the test cases and "run" has to exit with either 0 or 1; also depending on the interestingness. Except for the fact that it have to be Python the "API" of the script could easily be changed. They have to be Python scripts because this allows to use the "multiprocessing" module which is platform independent and supports everything that is needed to run the interestingness tests as separate process and to wait on any process to finish. > One thing I've wanted is a way to better specify the pass ordering, which is > kind of hacky. It would also be nice to allow users to customize the passes > using a little file. Also we'd like to customize the passes based on the > language that is being reduced. At the top-level passes are now represented in groups. This allows to make different configurations based on languages etc. Each group consists of three subgroups (first, main, last) which itself contain a list with passes. The priority is now implied by the position in the list. The downside is that the passes have to be repeated for each group but this layout makes it easy to take for instance a JSON file as input to define a custom group of passes (not implemented yet). > Regarding the passes, there's some pretty bad stuff in the regex-based > passes. These are basically the original C-Reduce code from the mid/late > 2000s. It would be great to find better ways to do these rewrites. I hope the nested regular expressions are now a bit nicer and maybe even faster. I had to write a custom parser for nested expressions as there seems to be no build-in approach. Currently it is more like a prototype -- though fully functional -- and I am sure it could be further improved. Both in terms of ease of use and performance. I am happy about any questions or comments. In the meantime I will try to keep my version up-to-date with the master branch of C-Reduce. Regards, Moritz
Re: [creduce-dev] [RFC] Switching from Perl to Python
Hi John, > Hi Moritz, this is cool. I've thought about the Perl vs Python issue a > number of times and basically I just do not love Python no matter how many > times I start writing it. On the other hand I can probably get over this. Oh, seems like a will have a hard time to convince you that Python is not too bad in the end. When I first used Python I was sceptical as well but after using it for some time it has become a convenient tool for tasks which require a little bit more than just a shell script. But I guess that is what you would use Perl for. > My guess is that the speedup you're seeing is mostly due to running fewer > passes, since in general CPython is pretty suckily slow compared to Perl. > Probably not a big issue for C-Reduce, however, which is almost always > bottlenecked by interestingness tests. Just be clear, I run the same passes for both version (Perl and Python). Because the Python version is not complete yet, I had to disable some of the passes in the Perl version. I also checked that both version produce the same reduced file in the end. It might be that not using the original test scripts but Python based ones have caused the difference. But as long as performance is not a main criterion for you I would spent more time in analysis this. > I do feel strongly that the abstraction boundary between the core and the > passes and the interestingness tests should be a strong one, probably a > process by default. I agree. And unless you would want to give up the parallel approach there is currently no other way to run multiple tests in parallel with Python than to launch processes. I first thought about threads but the "Global Interpreter Lock" prevent concurrent execution of Python code in multiple threads. > Anyway I need to think about it more and no doubt the other C-Reduce people > will have opinions. I'm open to moving to a different implementation of the > C-Reduce core, but not until the replacement is feature complete (and I'm > probably not going to have a lot of time to work on it myself, but I'm happy > to do code reviews). I would be willing to do the rewriting. Though instead of just "translating" the Perl code into Python I would suggest to think about potential changes to improve the maintainability and readability. Maybe you have something that you always wanted to change anyway but never had the time? I think that could be easily done in the style of small code reviews by splitting the work up into smaller chunks. > Keep in mind that the C-Reduce passes are not all equally useful and some > merging and removing of functionality can probably be done without hurting > the end results. If you want to remove something it would reduce the effort of rewriting but currently I think it shouldn't be necessary. And it is not much left to implement. Let's see if others are interested too but if I have some spare time I might already continue to port the last passes to see if there might be more difficulties than I expect. And even if the Python version is not used in the end I might find some bugs which can be fixed. (Due to a mistake in my Python implementation I noticed that the remove-pointer-pairs pass should have been called reduce-pointer-pairs -- now fixed ;-) Regards, Moritz > On 5/26/16 9:36 PM, Moritz Pflanzer wrote: >> Hi all, >> >> I am wondering if there might be interest in rewriting the C-Reduce core >> algorithm and the reduction passes in Python. Potential benefits could be: >> >> - I suspect more people are familiar with Python than with Perl >> - Python offers a lager set of features without the need to install >> additional modules (see below) >> - The implementation seems to be a bit simpler and cross-platform >> compatibility seems to be easier (see below) >> - Python is more actively maintained? (Here I am just guessing based on >> recent popularity) >> - A Python based implementation could lead to smaller run-times (see below) >> >> Feel free to add other points or to discuss about potential cons of >> switching. So far I could think of: >> >> - Some effort is required to do the rewriting >> - You guys might be more familiar with Perl? >> >> >> To push a little bit more in the direction of switching over I created a >> first proof of concept Python version and compared (most of) the included >> test between the existing Perl and my Python version. Because the Python >> version is not complete yet (see below) I had to disable a few passes to >> allow a fair comparison. And I ran only tests 0-3 and 6, 7 because 4 and 5 >> make use of KCC and Frama-C and I did not want to go through to much trouble >> setting e
[creduce-dev] [RFC] Switching from Perl to Python
Hi all, I am wondering if there might be interest in rewriting the C-Reduce core algorithm and the reduction passes in Python. Potential benefits could be: - I suspect more people are familiar with Python than with Perl - Python offers a lager set of features without the need to install additional modules (see below) - The implementation seems to be a bit simpler and cross-platform compatibility seems to be easier (see below) - Python is more actively maintained? (Here I am just guessing based on recent popularity) - A Python based implementation could lead to smaller run-times (see below) Feel free to add other points or to discuss about potential cons of switching. So far I could think of: - Some effort is required to do the rewriting - You guys might be more familiar with Perl? To push a little bit more in the direction of switching over I created a first proof of concept Python version and compared (most of) the included test between the existing Perl and my Python version. Because the Python version is not complete yet (see below) I had to disable a few passes to allow a fair comparison. And I ran only tests 0-3 and 6, 7 because 4 and 5 make use of KCC and Frama-C and I did not want to go through to much trouble setting everything up. ;-) (Running them wouldn't have been a problem, though.) My detailed results can be found here: https://docs.google.com/spreadsheets/d/1FIvuHr29X2T2H2wOrnGCU0BUM3NeQrvJY_GpKMVJRCA/edit?usp=sharing In short: On Linux my Python version takes only 62% of the time on average, on Windows there is not much of a difference. (This might be because the bottleneck on Windows is the process creation -- as opposed to forking on Linux -- and not the passes themselves.) On Linux the Perl variant used the original shell test scripts, for the Python variant I converted the tests to equivalent Python function. In both cases each test was run as a separate process, so I guess the comparison is fair. On Windows, since I could not run the shell scripts, both variants used the same Python scripts. Some words about the Python version. First, it can be found here: https://github.com/mpflanzer/creduce/blob/python/creduce/creduce.py - It took me about 10-20 hours to write this version -- hard to say how long exactly since I could always only work for short periods. I would estimate that it is about 70% complete with respect to the Perl version. - I have written it in Python3 as it offers some convenient features over Python2 and the recommendation is to start new work with Python3 anyway. - It does not use anything but the modules which come with the default Python installation (both Linux and Windows) - I think the largest missing piece are the passes that remove matched parentheses, braces etc. Python has no built-in functionality so a small custom parser would have to be written -- should not be to difficult - I have not yet figured out the best way to represent, load and execute the interestingness tests. Ideally I would like to have a base class from which each custom test could inherit. Each test would then be written in a separate Python script but dynamically imported into the C-Reduce script. Then it could be used as any other class. If that's not really feasible it is however no problem to just run them as independent scripts -- the same way like it is now in the Perl version. I think that is all I can report for now. Please let me know what you think about the idea or if you need some more information. I might have missed something in this writeup. Best regards, Moritz
Re: [creduce-dev] New planned release
Hi,
> We could use testing on Windows! Anyone have a machine handy? I have access
> to zero Windows machines right now. I think MS has free VM images these days
> but I probably don't have time to get one setup to the point where it can run
> C-Reduce.
I did some testing on Windows without any environment like Cygwin etc.
There are a few things that do not work currently or that behave not in the
same way as on Linux.
To build C-Reduce I used the Cmake build system. I already included the pull
request from vgvassilev (https://github.com/csmith-project/creduce/pull/87).
Things I noticed:
- Support to build "topformflat" from the sources is missing. I added it and
removed the configuration check if it can be found.
- Unifdef is not build with the Cmake system. I added it but it cannot be build
on Windows. It depends on "unistd.h" which does not exist for Windows.
- The install path for clang_delta etc. did not match the value that was
written to the config file ("bin" is missing at the end of the install
destination)
- Further, the install destinations of the Cmake system are different from the
Automake build. For instance, the Cmake system does not create a "libexec" dir
but puts everything in "bin". While this does not affect the execution I would
suggest to keep both systems as similar as possible.
After I made these small changes to the build system everything got installed
successfully. So in general building and installing on Windows works fine.
Then, when I started a reduction there where some other issues:
- Since unifdef could not be build its prerequisites could not be fulfilled and
I had to remove the pass from the list of all passes
- The pass_lines pass makes use of "grep" to remove blank lines after
topformflat has been run on the test case. grep is not available on Windows and
thus all passes failed. Also the was a problem if the pass to the topformflat
executable contained whitespaces. I replaced the grep command with loop over
the file after topformflat has finished. Maybe not the best way but it allowed
me to test further.
- Due to some changes in the delta_test mechanism the wait_helper function
returned a wrong result which basically made all test case interesting. After I
found the issue it was easy to fix though.
And that's all I have to complain about. ;-) With all the changes C-Reduce can
be run on Windows without any additional environments.
All my changes are currently is this branch
https://github.com/mpflanzer/creduce/tree/topformflat_cmake but it's a bit
unorganised so will not create a pull request for the moment.
But if we decide how to handle the remaining issues I would be willing to
prepare a clean patch which you could merge.
Regards,
Moritz
Re: [creduce-dev] New planned release
Hi, I have a Windows machine at hand. I have already everything set up and working for an older version of C-Reduce but I might find some time over the weekend to update to the latst version. This is something I wanted to do anyway. There were just other more important things to do so far. :/ Is there anything in particular you want to have tested? Regards, Moritz On 5 Apr 2016 15:48, John Regehr wrote: > > Thanks Eric! I have time to help in the near future. > > There was some C++ hacking I was trying to get done for this release but > since I'm having trouble getting back to it, let's push this out soon. > > We could use testing on Windows! Anyone have a machine handy? I have > access to zero Windows machines right now. I think MS has free VM > images these days but I probably don't have time to get one setup to the > point where it can run C-Reduce. > > John > > > On 4/5/16 4:44 PM, Eric Eide wrote: > > Martin Liška writes: > > > >> I would like to ask you when do you plan a next release? I would like > >> to make a new package for openSUSE that incorporates a commit that > >> implements multiple input files. > > > > "Very soon." Let me say, in the next two weeks. > > > > We are working to make a new release that is compatible with LLVM 3.8. It > > will include other work to date, including multi-file reduction. > > > ! > I am working on the release now --- see recent commits --- but I haven't had > > > much time to work on it in the last little while. (FWIW, I was hoping > that if > > I waited a bit, the current problem the Travis-CI build would > fix itself, but > > that hasn't worked out.) Most of what remains to be > done is testing on various > > platforms/configs and a review of the > included documentation, so we are close. > > > > Thanks --- > > > > Eric. > >
