Hi folks,
I need help debugging a strange test failure in pylint-django that started
happening after the upgrade to pylint/astroid 2.0.
For all I can tell we've already adjusted the code and this failure is happening
randomly. pylint-django's test used uses the test_functional.LintModuleTest
class from pylint (I know it isn't public interface) and executes against input
files comparing the linter results with expected.
The error that we get is:
E Failed: Wrong results for file "func_noerror_foreignkeys":
E
E Unexpected in testdata:
E 27: no-member
The strange thing is that on one test job in Travis this passed with Python 3.4
and Django 2.0 (see job #447.3):
https://travis-ci.org/PyCQA/pylint-django/builds/407129036
and on the next commit this failed on Python 3.4/Django 2.0 (see #448.3):
https://travis-ci.org/PyCQA/pylint-django/builds/407800899
however it passed on 3.5 + Django 2.0.
The two test jobs use the same code (minor differences in commits due to
squash/conflicts).
When the linter is executed manually against the offending file it produces a
10/10 score.
If I modify the test suite like so:
--- a/pylint_django/tests/test_func.py
+++ b/pylint_django/tests/test_func.py
@@ -43,6 +43,8 @@ def get_tests(input_dir='input', sort=False):
suite = []
for fname in os.listdir(input_dir):
+ if fname.find('func_noerror_foreign') == -1:
+ continue
if fname != '__init__.py' and fname.endswith('.py'):
suite.append(test_functional.FunctionalTestFile(input_dir, fname))
then I get only a subset of the tests executed (including the offending one) and
this time it passes.
Anyone seen such erratic behaviour ? I think this is a flaky test but I have no
idea how to approach debugging that.
For the record this is blocking us from releasing a new version of pylint-django
so any help is appreciated.
--
Alex
_______________________________________________
code-quality mailing list
code-quality@python.org
https://mail.python.org/mailman/listinfo/code-quality
