Update for those interested, the solution is here with explanation: https://github.com/PyCQA/pylint-django/pull/183/commits/538546044ac5a9785dc85369394d2de8c78349fc
Cheers, James On Tue, 24 Jul 2018, at 10:23 PM, Alexander Todorov wrote: > Hi folks, > I need help debugging a strange test failure in pylint-django that started > happening after the upgrade to pylint/astroid 2.0. > > For all I can tell we've already adjusted the code and this failure is > happening > randomly. pylint-django's test used uses the > test_functional.LintModuleTest > class from pylint (I know it isn't public interface) and executes > against input > files comparing the linter results with expected. > > The error that we get is: > > E Failed: Wrong results for file "func_noerror_foreignkeys": > E > E Unexpected in testdata: > E 27: no-member > > The strange thing is that on one test job in Travis this passed with Python > 3.4 > and Django 2.0 (see job #447.3): > https://travis-ci.org/PyCQA/pylint-django/builds/407129036 > > and on the next commit this failed on Python 3.4/Django 2.0 (see #448.3): > https://travis-ci.org/PyCQA/pylint-django/builds/407800899 > > however it passed on 3.5 + Django 2.0. > > The two test jobs use the same code (minor differences in commits due to > squash/conflicts). > > When the linter is executed manually against the offending file it produces a > 10/10 score. > > > If I modify the test suite like so: > > --- a/pylint_django/tests/test_func.py > +++ b/pylint_django/tests/test_func.py > @@ -43,6 +43,8 @@ def get_tests(input_dir='input', sort=False): > > suite = [] > for fname in os.listdir(input_dir): > + if fname.find('func_noerror_foreign') == -1: > + continue > if fname != '__init__.py' and fname.endswith('.py'): > suite.append(test_functional.FunctionalTestFile(input_dir, > fname)) > > then I get only a subset of the tests executed (including the offending > one) and > this time it passes. > > > Anyone seen such erratic behaviour ? I think this is a flaky test but I > have no > idea how to approach debugging that. > > > For the record this is blocking us from releasing a new version of > pylint-django > so any help is appreciated. > > -- > Alex > > _______________________________________________ > code-quality mailing list > code-quality@python.org > https://mail.python.org/mailman/listinfo/code-quality _______________________________________________ code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality
