Re: [Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
---+--
 Reporter:  Jakub Kulík|Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  3.2
 Severity:  Normal |   Resolution:  invalid
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by Jakub Kulík):

 * resolution:  needsinfo => invalid


Comment:

 I am sorry, I just found the issue and it's entirely on our side.

 For posterity: Django test suite can be fragile in a sense that when one
 test fails, many unrelated may follow, and that's what happened here.
 `test_incorrect_timezone` is the only one that is actually failing and the
 cause is our internal Solaris issue with `/usr/share/zoneinfo`. Once
 patched, the test suite is again 100% green.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.80dc30410f4464301c877f1fbfcd21fe%40djangoproject.com.

Re: [Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
---+--
 Reporter:  Jakub Kulík|Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  3.2
 Severity:  Normal |   Resolution:  needsinfo
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by Mariusz Felisiak):

 Replying to [comment:4 Jakub Kulík]:
 > Unfortunately, it seems so... Do you have any pointers/ideas on how I
 should debug this? I'll try to find and fix the issue.

 It would help to find test classes (or even a set of tests) that are
 reproducing this issue without running the entire test suite. Maybe
 `test_runner` is enough.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.e40f5164062e38bc61a7aa5daec09629%40djangoproject.com.

Re: [Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
---+--
 Reporter:  Jakub Kulík|Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  3.2
 Severity:  Normal |   Resolution:  needsinfo
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by Jakub Kulík):

 Unfortunately, it seems so... Do you have any pointers/ideas on how I
 should debug this? I'll try to find and fix the issue.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.1670f3cf068d9d961ff02f8f0116938f%40djangoproject.com.

Re: [Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
---+--
 Reporter:  Jakub Kulík|Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  3.2
 Severity:  Normal |   Resolution:  needsinfo
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by Mariusz Felisiak):

 Thanks for extra details, unfortunately I still cannot reproduce these
 failures so it must be something Solaris-specific.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.ffadc486104d4c387fb73ae1b05401d3%40djangoproject.com.

Re: [Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
---+--
 Reporter:  Jakub Kulík|Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  3.2
 Severity:  Normal |   Resolution:  needsinfo
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by Jakub Kulík):

 I see this issue only with Django 3.2 on SPARC (with both Python 3.7 & 3.9
 and both with and without `--reverse`). Intel in this case is without
 issues, I also don't see this in Django 3.1 or 3.0 with all combinations
 of Python runtime (3.7 and 3.9), architecture (SPARC and Intel), and order
 (`--reverse` or not).

 In each case, tests are executed with the following command:
 `/usr/bin/env LC_ALL=en_US.UTF-8 /usr/bin/python3.x ./runtests.py
 --settings test_sqlite --parallel=1 --verbosity=3 (--reverse)`

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.9af0862188bbddc9b0f922628dd1ea39%40djangoproject.com.

Re: [Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
---+--
 Reporter:  Jakub Kulík|Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  3.2
 Severity:  Normal |   Resolution:  needsinfo
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by Mariusz Felisiak):

 * status:  new => closed
 * resolution:   => needsinfo


Comment:

 I cannot reproduce these failures, all tests works for me when run in
 reverse, see also https://djangoci.com/job/main-reverse/. Can you provide
 a subset of these that causes failures for you? flags passed to
 `runtests.py`, Python version, database, etc.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.0a459cd23c49baad627c7296422e0310%40djangoproject.com.

Re: [Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
---+--
 Reporter:  Jakub Kulík|Owner:  nobody
 Type:  Bug|   Status:  new
Component:  Testing framework  |  Version:  3.2
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by Jakub Kulík):

 * Attachment "reverse-failures.txt" added.

 Test failures when running with --reverse

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.e23636e8bb2215401ceb31eccf2937c3%40djangoproject.com.

[Django] #32746: Unexpected test failures in 3.3.2 when running the whole testsuite

[Django]

#32746: Unexpected test failures in 3.3.2 when running the whole testsuite
-+
   Reporter:  Jakub Kulík|  Owner:  nobody
   Type:  Bug| Status:  new
  Component:  Testing framework  |Version:  3.2
   Severity:  Normal |   Keywords:
   Triage Stage:  Unreviewed |  Has patch:  0
Needs documentation:  0  |Needs tests:  0
Patch needs improvement:  0  |  Easy pickings:  0
  UI/UX:  0  |
-+
 We recently introduced Django 3.3.2 to Solaris (with Python 3.9) and we
 are hitting unexpected test failures when running the full test suite on
 SPARC. When I rerun those tests one by one however, they are successful.
 My guess is that parts of the test suite are not independent and affect
 each other.

 Errors we are seeing are (although I am not sure that they are always the
 same and they changed since 3.3.1):
 {{{
 ERROR: test_output_normal (test_runner.test_debug_sql.TestDebugSQL)
 ERROR: test_output_verbose (test_runner.test_debug_sql.TestDebugSQL)
 ERROR: test_setupclass_exception (test_runner.test_debug_sql.TestDebugSQL)
 ERROR: test_lazy_base_url_init (file_storage.tests.FileSystemStorageTests)
 FAIL: test_incorrect_timezone (settings_tests.tests.SettingsTests)
 FAILED (failures=1, errors=4, skipped=1146, expected failures=4)
 }}}

 Running the test suite with `--reverse` results in many more failures
 (attached).

 This happens on SPARC only, Intel test results are clean (both with and
 without `--reverse`). Our other Django 2.2.21 with Python 3.7 is clean as
 well on both platforms.

 I am not sure how to continue/investigate the issue further, but I am
 happy to do so if guided a little.

 I have yet to try Django 3.1 or Python 3.7 (but runtime is highly unlike
 the problem).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/051.415e36da2713ec2ff41662e57642cd45%40djangoproject.com.