https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7982
--- Comment #17 from Sidney Markowitz <sid...@sidney.com> --- Answering in reverse order: (In reply to Henrik Krohns from comment #16) > we should stop SATest.pm from copying any cf from trunk/rules at all That's something I was already considering while working on this. It might still be reasonable to make it easy to test against the current rules instead of the packaged subset, but that would be better as an explicit option rather than something that happens when you don't pay attention to the test environment. (In reply to Henrik Krohns from comment #15) > Does the fix make sense though? As mentioned, isn't it a advantage that "make > test" actually tests that current trunk/rules does what it's supposed to > correctly? I have mixed feelings about this. I think that tests should be as self-contained as possible. If they depend on current rules, they can break when rules change. They can even break silently by not failing when they should. The advantages of 01_test_rules.cf are 1) It documents the smallest set of rule dependencies that are needed for the tests; and 2) It encourages one to write tests so that they don't depend on rules, because any new dependencies will require adding something to 01_test_rules.cf. The disadvantages are that it freezes those rules as used in the tests. One thing about doing it this way, is that running make test from a trunk checkout directory will see all the current rules from the rules directory. Running it from an install package directory will have the stripped down rules directory and will test the more self-contained frozen version in 01_test_rules.cf. So tests can be run both ways, which should detect different problems. I don't have any concern that users will be confused by the files, they'll never see it. I do feel off about the distribution and tests depending on an arbitrary snapshot of the entire set of rules as they are when the release is tagged. Using only a small subset of rules in 01_test_rules.cf is more controlled and easier to keep track of. Along those lines, I would really like it if someone could look at sql_based_welcomelist.t and figure out how to make it dependent on much fewer rules. As far as I can tell it has test cases that are designed to trigger different rules and it checks that the expected results get processed. What it is testing has nothing to do with those specific rules, they are just used to see if the expected strings are generated. But each such rule adds a requirement for one or more rule files to be available. Anyway, what I want to do is continue seeing what has to go into 01_test_rules.cf to allow the tests to succeed. Once I do that, or hit a roadblock, we can compare that to the option of copying all the rules into the install. I already have a patch that does that, so we can decide then. -- You are receiving this mail because: You are the assignee for the bug.
