Change by David Bolen :
--
nosy: +db3l
___
Python tracker
<https://bugs.python.org/issue45806>
___
___
Python-bugs-list mailing list
Unsubscribe:
David Bolen added the comment:
So I'm guessing something is just borderline under 3.9 on Windows.
In some manual testing with a standalone build of 3.9 so far for me:
-m test.test_pickle always succeeds (executed directly)
-m test test_pickle always fails (executed via
David Bolen added the comment:
I don't know if this is a buildbot, test or 3.9-specific issue but this commit
appears to have introduced a permanent initial failure (but success on retry)
in test_pickle on both Windows 10 3.9 builders. First failure for my builder
at https
David Bolen added the comment:
Oddly, it turns out it's the "--junit-xml" parameter during buildbot tests that
is making it reproducible there.
Pass: python_d -m test.regrtest test_ssl
Fail: python_d -m test.regrtest --junit-xml out.xml test_ssl
The latter consistently
David Bolen added the comment:
The win10 buildbot appears to have this failure consistently (100%) on the 3.10
branch. The first such failure appears to be yesterday
(https://buildbot.python.org/all/#/builders/600/builds/79), following commit
da8097aaf5a55c23f5b5ddbeffc2d90d06e00d93 - GC
David Bolen added the comment:
No longer needed after commit c368ce74d2c9bcbf1ec320466819c2d4768252f7
--
resolution: -> out of date
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
David Bolen added the comment:
Based on further information in issue #27129 as well as issue #43719 it appears
a source of the problem prompting this fix was a failure to update the magic
number in the original commit for #27129.
The windows clean script does still leave more artifacts than
David Bolen added the comment:
Dennis, just to make sure I wasn't too literal, some of the __pycache__ folders
are in subdirectories (at least in Tools), so just to double check, you did a
recursive search beneath Tools and Parser right
David Bolen added the comment:
Terry, it's not clear to me if this is the same issue (at first blush it
appears different) but could you see if you have any pyc files in __pycache__
folders within the Tools or Parser directories in your tree that could be from
prior builds? If so, remove
David Bolen added the comment:
I don't think reverting the commit at this point would necessarily be helpful.
While it might fix some systems, it could newly break anyone who happened to do
their first build since the commit was in place.
I didn't want to bug anyone over the weekend
Change by David Bolen :
--
keywords: +patch
pull_requests: +23904
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/25157
___
Python tracker
<https://bugs.python.org/issu
David Bolen added the comment:
Something like this is a quick 'n dirty minimal fix - at least it seems to
solve the problem that arose in issue #27129 on the Win10 buildbot:
--- a/Tools/buildbot/clean.bat
+++ b/Tools/buildbot/clean.bat
@@ -11,6 +11,8 @@ call "%pcbuild%\build.bat" -
David Bolen added the comment:
I've opened issue #43709 for fixing the buildbot clean script under Windows.
It needs to clean the Tools and Parser trees, not just Lib (and there are a few
other folders involved besides clinic)
--
___
Python
New submission from David Bolen :
The Tools\buildbot\clean.bat script used on Windows only removes pyc/pyo files
from the Lib tree, leaving some files beneath Tools (clinic and peg_generator)
and Parser (asdl). This can cause failures following commits that affect those
files
David Bolen added the comment:
I'm out of time for a bit, but it appears that the root issue is old pyc files
in Tools/clinic/__pycache__ that aren't removed during a clean process, and
appear to be the source of all of the errors. Manually pruning that folder
fixes things.
I believe
David Bolen added the comment:
Ah, Victor, that helps. I was having trouble reproducing the problem on a
different system. I was suspecting a small difference in compiler version, but
I hadn't considered it being because I started fresh.
>From what I can see, a particular build tree
David Bolen added the comment:
Unfortunately, not at the moment - what's in the buildbot log is what's
available. The RTL assertion aborts the process.
The tests involved (such as test_clinic) do seem reproducible in a few separate
tries, though again, all they do is terminate
David Bolen added the comment:
Note that this commit appears to be causing exceptions for the Win10 buildbot,
failing the PyCode_Addr2Line assertion in codeobject.c line 1252.
The assertion seems to pop up at differing points during each test run, but the
builder has yet to complete a full
David Bolen added the comment:
In lieu of the patch in #25191, what about a pair of skips to deal with the
issues at hand without killing the test entirely? I'm including OpenBSD since
those issues were closed in favor of this one, and am assuming that skipping
there is also appropriate
David Bolen added the comment:
I don't have much of a horse in the race, but since the test has historically
been skipped on Windows, and the test hasn't and doesn't work on Windows,
modifications to restore the skip behavior seem reasonable to me. The trigger
for this issue was Windows
David Bolen added the comment:
The test has also begun failing on the Win10 buildbot (after updating to 20H2
from an older 1803).
--
nosy: +db3l
___
Python tracker
<https://bugs.python.org/issue37
David Bolen added the comment:
The win10 buildbot is green again, so I think this can be closed.
--
___
Python tracker
<https://bugs.python.org/issue43
Change by David Bolen :
--
keywords: +patch
pull_requests: +23510
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/24739
___
Python tracker
<https://bugs.python.org/issu
David Bolen added the comment:
Yes, that was the idea (revert the PyDEBUG condition only); it sounds like
everyone is on the same page.
I've been doing buildbot duties only for a while (no code contributions since
long before the current process), so there may be a short delay while I
David Bolen added the comment:
Steve, where is that configured? If reducing that further would resolve the
crashes while retaining ceval debugging, maybe that's a reasonable trade-off,
though based on my testing, reverting still seems simpler.
Right now the debug build on the buildbot
David Bolen added the comment:
I hadn't tested release mode earlier, since the commit only removed the pragma
in debug builds, but I just built a release build with the commit in place on
the worker and it seems fine.
So barring stack changes (enlarging or improving usage) it appears
David Bolen added the comment:
I don't think it's actually any change in ceval per se, or any new buffers,
just how the compiler code generation has changed due to this commit.
Based on some local testing, the triggering issue is the exclusion of the
optimization pragma entirely in debug
Change by David Bolen :
--
nosy: +db3l
___
Python tracker
<https://bugs.python.org/issue43166>
___
___
Python-bugs-list mailing list
Unsubscribe:
David Bolen added the comment:
I was wondering if there was any update on whether or not this new behavior can
be corrected?
I was attempting to review a buildbot failure today and it's actually pretty
tough to "race the refresh" when trying to review the build step
David Bolen added the comment:
This change to the 3.8 branch appears to be consistently failing on the Windows
7 buildbot (first failing build at
https://buildbot.python.org/all/#/builders/270/builds/126).
It's all failures in the new test_configure_custom_copy and
test_map_custom_copy
David Bolen added the comment:
I'm guessing the warning appears odd as we're seeing a thread shutdown data
race. The message is produced by threading_cleanup in
support/threading_helper.py, and it appears that between the first and second
lines one of the initially dangling threads goes
David Bolen added the comment:
I've been seeing failures on the Win10 buildbot 3.x branch that seem to
correlate with the timing of this change - could there be some further work
needed on Windows? Or, if it's a test-only artifact and the warnings are
innocuous, something to ignore
David Bolen added the comment:
It looks like there was an underlying asyncio.recv_into bug that was the likely
root issue here. It's recently been fixed in bpo-41467, and test_asyncio is
passing on at least the Win10 buildbot.
--
___
Python
David Bolen added the comment:
Just for the record, this fix also appears to have resolved the issue with the
Win10 buildbot from bpo-41273, which was related to the same unraisable
exceptions mentioned in bpo-38912.
--
nosy: +db3l
___
Python
David Bolen added the comment:
First, I also no longer see the error with a local PR 21446 build on the Win10
buildbot.
As for timing, I believe policy is to revert, but in my view it can probably
depend on how short "a bit" is for the fix. We've been in this state for 4-5
da
David Bolen added the comment:
I can at least confirm that this commit is the source of the issue on the
Windows 10 buildbot. Interactively, it fails every time with it in place
(unlike the buildbot which has had the occasional success) and passes reliably
with it reverted.
The error
David Bolen added the comment:
So a script I use on the buildbot intended to prevent hanging on assertions was
failing to work in this particular case, which I've corrected. That changes
the failure mode for new Win10 buildbot runs.
The test in question (test_close_stdin) still fails
David Bolen added the comment:
It appears the recent commit is causing a CRT exception dialog in
test_close_stdin (test_repl.TestInteractiveInterpreter). The dialog can't get
answered, which is what leads to the eventual timeout.
The assertion is "_osfile(fh) & FOPEN" fr
David Bolen added the comment:
I haven't had a chance to look too deeply, but the final set of commits
(starting with fa7ab6aa) appear to be the common thread with all branches now
failing with timeout exceptions in test_repl on the Windows 10 buildbot.
The first instance was in the 3.x
Change by David Bolen :
--
nosy: -db3l
___
Python tracker
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by David Bolen :
--
nosy: -db3l
___
Python tracker
<https://bugs.python.org/issue33608>
___
___
Python-bugs-list mailing list
Unsubscribe:
David Bolen added the comment:
Ok, I can confirm that the updated PR 17774 test passes under Windows (and
still cleans up after itself).
--
___
Python tracker
<https://bugs.python.org/issue27
David Bolen added the comment:
I'd be happy to test an updated PR 17774 on a Windows builder, but I don't
actually see any change yet. It still appears to hold the earlier
NamedTemporaryFile with mode='w' change from a few days ago
David Bolen added the comment:
The issue appears to be the temporary flag (O_TEMPORARY) that is used under
Windows with delete on close temporary files. That appears to prevent any
separate access to the file by anyone else including obtaining another
reference in the same process
David Bolen added the comment:
I think fixing the underlying pty issue should certainly be the goal, but the
question is whether the process group change should remain active in the
meantime, as its presence is causing a regression in the tests. I think such
cases in the past are usually
Change by David Bolen :
--
nosy: +db3l
___
Python tracker
<https://bugs.python.org/issue38547>
___
___
Python-bugs-list mailing list
Unsubscribe:
David Bolen added the comment:
I can recreate this manually by running regrtest.py against test_pty. Crashes
with any "-j#" option, but fine when run sequentially. Removing the process
group change avoids the crash.
With the process group change in place, the trigger poi
David Bolen added the comment:
I don't know for sure that this is the cause but both 3.x builds following this
commit on my bolen-ubuntu worker (Ubuntu 18.04.3) have had test_pty crash in
the first attempt, with the retry succeeding. For example
https://buildbot.python.org/all/#/builders
David Bolen added the comment:
I've confirmed the partial read with some local modifications, and the failures
are always split between time stamp and value:
Warning -- Failed to parse typeperf output: '"10/02/2019 17:42:26.229"'
0.0
Warning -- Missing first field: ,"0.
David Bolen added the comment:
Oh, I agree it's just a warning, and I suspect few people look into warnings,
but since it's not from an actual test, I'm not sure the overall build should
be flagged.
The manual typeperf looks fine, but there's no way I could tell visually how
the I/O
David Bolen added the comment:
Just an FYI that this change is generating warnings on my Windows 10 buildbot
with some regularity about a failure to parse testperf output, such as:
Warning -- Failed to parse typeperf output: '"10/01/2019 07:58:50.056"'
from https://buildbot.pyth
David Bolen added the comment:
The new test_check_c_globals.ActualChecks test is failing with an "unexpected
success" on the bolen-ubuntu buildbot (under Ubuntu 18.04.3). I can reproduce
the failure in a manually built tree.
--
n
David Bolen added the comment:
Yeah, I think you're right.
It looks like without an explicit code, it won't propagate the result as the
exit code of cmd itself for those cases where cmd does exit (which would
include the buildbots
David Bolen added the comment:
I've been investigating issues with test failures on my Windows buildbots
seemingly not showing up in the master's web interface (but just showing
warnings), and it appears likely due to this change.
For example, test_urllib (a test problem from issue 36948
David Bolen added the comment:
Oh, and just for historical purposes, it looks like the root cause was that the
nturl2path.pathnametourl forces an uppercase drive letter. So that's where the
inconsistency in the test got introduced.
--
___
Python
David Bolen added the comment:
Yes, PR 13476 tested locally on the Win10 builder resolves the error.
--
___
Python tracker
<https://bugs.python.org/issue36
David Bolen added the comment:
Since this patch was introduced to the 3.x branch my Windows 7 and 10 buildbots
have been failing in test_urlopener_retrieve_file. See
https://buildbot.python.org/all/#/builders/3/builds/2661 for the first such
failure on the Win10 worker.
The problem
David Bolen added the comment:
I think I'm the wrong David for this...
--
___
Python tracker
<https://bugs.python.org/issue36749>
___
___
Python-bugs-list mailin
David Bolen added the comment:
I should mention that a high level of test parallelism on the part of my worker
might have be a contributing factor in this most recent case.
The worker was recently upgraded to a faster 4-core VM, but with limited I/O.
In a test run the test processes
David Bolen added the comment:
Ok, I've resolved this, and the linker errors did actually point at the root
issue. They show trying to link extensions against
/usr/local/lib/libpython2.7.a which was my test static build of 2.7.16.
Arguably this seems an issue in the buildbot build process
David Bolen added the comment:
Yes, it appears most likely to be the worker environment. I did upgrade the
kernel in between builds 250 (good) and 251 (bad), but reverting that still
fails, even with the same commit as in build 250.
My current suspicion is that a test I did recently
David Bolen added the comment:
I just noticed that my last message referenced the wrong commit. My test
failures were against commit f13c5c8b9401a9dc19e95d8b420ee100ac022208 (the same
as Victor).
--
___
Python tracker
<https://bugs.python.
David Bolen added the comment:
Eric, I'm also seeing the same Win7 and Win10 worker failures with commit
b75b1a350 as last time (test_multiprocessing_spawn on both, and test_io on
Win7).
For test_multiprocessing_spawn, it fails differently than Victor since no core
file is generated, but I
David Bolen added the comment:
Ok, I've verified that on a Win7 system with SP1 but without KB2533625 I get
the expected block screen at startup.
On my worker (SP1 with KB2533625) it proceeds to the regular installation main
dialog.
I'm attaching a copy of the install log in the blocking
David Bolen added the comment:
I can help with a Win7 test of the installer, but my currently available
systems are all 32-bit - any chance at a 32-bit version of the installer?
--
___
Python tracker
<https://bugs.python.org/issue36
David Bolen added the comment:
I just wanted to acknowledge that this was breaking on my Windows 7 builder
(with a bad DLL load parameter in both pythoninfo and test steps).
It looks like I was missing the required KB2533625 (the machine is mostly a
virgin SP1 install), so I've installed
David Bolen added the comment:
If I can help with testing or builder access or anything just let me know. It
appears that I can pretty reliably trigger the error through just the
individual tests (test_daemon_threads_shutdown_std{out,err}_deadlock) in
isolation, which is definitely less
David Bolen added the comment:
I suspect changes for this issue may be creating test_io failures on my windows
builders, most notably my x86 Windows 7 builder where test_io has been failing
both attempts on almost all builds. It fails with a lock failure during
interpreter shutdown
David Bolen added the comment:
Ah, got it (and see the pipelines comment by Steve).
Jeremy, I suspect you might actually be able to restart the most recent 3.6
builds on my builders since you were the committer. It changed in Sep to only
allow python-core users and the "owner" of
David Bolen added the comment:
Oh, it's not the installation itself, I'm just wondering if allowing a newer
version is ok too?
Of course, it doesn't preclude expanding the build script in the future, so
I've installed 15063 to both Win8/10 workers.
I don't currently have access to restart
David Bolen added the comment:
Hmm, VS2015 started as a full installation (with UI), probably right from its
initial release. The build tools only installation (v141) is for VS2017.
Best I can tell I'm at update 1 - my update version in the registry is
14.0.24720 (plus I have a .1
David Bolen added the comment:
(and the working log)
--
Added file: https://bugs.python.org/file47986/msbuild-win10-good.log
___
Python tracker
<https://bugs.python.org/issue35
David Bolen added the comment:
> So before the change, the 16299 SDK wasn't being detected either, but perhaps
> the 10240 one was?
So I'm just confused
It does seems likely that 10240 of the UCRT was being used (based on the
attached msbuild logs). Howevr, the UCRT w
David Bolen added the comment:
(sorry for the rapid updates)
I'm also fairly sure that none of my workers have update 3 for VS2015. They do
however all have VS2017 - but I think VS2015 still gets picked for 3.6 if both
are present, right? So that's another variable, in that my workers
David Bolen added the comment:
Oh, since my reading comprehension must be low today, it appears like Jeremy
actually had a closer situation previously as I'm in now, with a later (not
older) version of the SDK that wasn't in the list.
Which is interesting, since he got an error about
David Bolen added the comment:
Well, correct me if I'm wrong, but installing 15063 would then match one of the
checks, and become the selected SDK, so be expected to work fine, right?
I think (not sure) that the issue with my Win8/10 workers is they only have the
later 16299. So
David Bolen added the comment:
I'm not that familiar (ok, at all) with the build process configuration, but in
looking at the changes to python.props, it appears to now enforce the minimum
in the build process regardless of whether it is found, whereas before the
build tool was allowed
David Bolen <db3l@gmail.com> added the comment:
For my buildbots, I suspect win8/win10 should be ok at this point. Among any
other changes their local disk appears to be an SSD now, which makes a big
difference. I observed steady 250-300MB/s write I/O for the duration of the
mma
David Bolen <db3l@gmail.com> added the comment:
The win7 builder isn't on Azure, so changing host type isn't an option at the
moment. For my part, I'd prefer removing the largefile tests for that one
rather than increasing timeout. The tests generate a huge amount of I/O, so my
David Bolen <db3l@gmail.com> added the comment:
I have migrated the win8 and win10 builders to a different machine type on
Azure, which seems to have restored more reasonable performance for the tests,
at least in the first set of builds. Assuming that continues to hold true, it
David Bolen <db3l@gmail.com> added the comment:
A longer timeout might be another workaround, but for myself, I tend to favor
Zachary's original suggestion of eliminating largefile tests for the moment as
simplest and most robust. It would also reduce the overall percentage of test
David Bolen <db3l@gmail.com> added the comment:
Disk space seems unlikely as a reason - the local disk is 60GB on each
buildbot, of which at least 50GB+ is generally free. So that shouldn't be a
problem (the WinXP/Win7 buildbots operate with far less, like 5-10GB).
I had also cons
David Bolen <db3l@gmail.com> added the comment:
Nothing for my part, but there was an Azure "maintenance" on 4/16 where both
the Win8 and Win10 buildbots were migrated to "newer" hardware (the first
change to their physical machine for a really long time). No wor
David Bolen <db3l@gmail.com> added the comment:
Apologies if this is obvious and already in progress, but the issue with
Windows and this change appears to be that the "utf-8.file" file is being
treated as text rather than binary.
So the line ending is expanding to CRLF.
David Bolen <db3l@gmail.com> added the comment:
After some further testing, it does not appear to be configure related but
something environmental (maybe MACOSX_DEPLOYMENT_TARGET) - using the same
configure options manually as build-installer seems ok.
For what it's worth, since th
David Bolen <db3l@gmail.com> added the comment:
This commit appears to have broken OSX installer builds using the
build-installer.py script (as in the last two attempts on the bolen-dmg-3.x
builder), confirmed with a bisection from the first failing worker build
(covering the
David Bolen <db3l@gmail.com> added the comment:
Yeah, I'm not sure if I can do anything on my side. It looks like it's an
issue with the build patch (issue30487, commit d8d6b91, assuming that's what
Zachary is referring to). The blurb complaint is about something that appears
David Bolen <db3l@gmail.com> added the comment:
Ok, so rc.exe appears truly not to be found when the test runs. The binary is
a bit buried in the Windows Kit directory tree, and I'm guessing something is
off after the upgrade (I'm not sure where it was in the tree before). I
ma
David Bolen <db3l@gmail.com> added the comment:
This seems to correlate with my upgrading to the latest Win10 SDK on those
workers (Steve pointed out I was still getting ucrt warnings during
compilation). So they both jumped from like 10240 up to 16299. But that's all
I c
David Bolen <db3l@gmail.com> added the comment:
Sounds good. I must admit I hadn't actually gone looking for a standalone
installer yet, but just assumed it would exist.
The Win10 worker now has the .NET 3.5 feature installed (which also installed
2.0 and 3.0). Builds seem fin
David Bolen <db3l@gmail.com> added the comment:
Sure, I can certainly do that.
Does "basically a requirement" mean it should have been there all along (with
the VC9 installation), or just that trying to use VC9 on a recent system like
Win 10 safely requires it installe
David Bolen <db3l@gmail.com> added the comment:
I believe I've identified the issue, and put a workaround in place.
The 986b7ffc commit somehow affected which binary of mt.exe is being run (for
version 5.2.3790.2076, 2005), changing from:
C:\Program Files\Microsoft SDKs\Windows\v6.
David Bolen added the comment:
Antoine, yes. Send me your public key (db3l.net at gmail) and I'll set it up.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
David Bolen added the comment:
In running the test under a local build, the issue is very repeatable, but I
believe it's actually due to slow process startup rather than a quick exit.
That is, adding a brief sleep after process creation and just before the
Request() call seems to fix
David Bolen added the comment:
Hmm, I wonder if this is another race condition similar to issue 8458?
I think that was thought to be related to the subprocess exiting quickly, in
which case the question might be why that might happen more so than the actual
descriptor error.
BTW, Victor
David Bolen added the comment:
Yeah, there were a few transient build errors while I was getting the buildbot
back online. My initial attempt to fix this ticket involved using the VS 2015
installer which turned out very badly, so I had to fall back to an older VM
image and start over
David Bolen added the comment:
Ok, I believe I've got the latest ucrtbased now on the Win7 buildbot
(validate_ucrtbase.py reports it as 10.0.15063.137).
It appears as if my Win8 and Win10 buildbots also have the older dll, so I'm
guessing they're just fast enough or lucky enough not to run
David Bolen added the comment:
Ok, this should be resolved. I opted to be a bit more conservative and only
upgraded to sqlite3 3.8.3.1, which is the oldest version still in use for the
dmg installer packages (for 2.7). I restarted the most recent 3.x build and
while it's not completely done
David Bolen added the comment:
The test appears to pass against a local test build of sqlite3 3.18.0, so I'll
probably just plan on updating the slave to that later tonight. Unless anyone
is interested in figuring out why it fails with 3.6.11 and not 3.18.0 (e.g.,
does the test have a higher
David Bolen added the comment:
No recent changes to the buildbot (I think the last was in September to bump
the local python used to run the slave/build installer to 2.7).
The default system sqlite (/usr/lib) is 3.0.8.6, so extremely old, but 3.6.11
is in /usr/local/{lib,include} from early
1 - 100 of 264 matches
Mail list logo
