[issue45055] Fresh build on Windows fails the first time for zlib.c

[Łukasz Langa]

Łukasz Langa  added the comment:

Looks like we can lose this with fixes merged for 3.9 - 3.11.

--
nosy: +lukasz.langa
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[miss-islington]

miss-islington  added the comment:


New changeset 50c0551f97f86bbf9f659e1bbe78a14df7754cbe by Miss Islington (bot) 
in branch '3.9':
bpo-45055: Add retry when downloading externals on Windows (GH-28399)
https://github.com/python/cpython/commit/50c0551f97f86bbf9f659e1bbe78a14df7754cbe


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[miss-islington]

miss-islington  added the comment:


New changeset ee41d01326ddf48c411c019a4e63343668ebd829 by Miss Islington (bot) 
in branch '3.10':
bpo-45055: Add retry when downloading externals on Windows (GH-28399)
https://github.com/python/cpython/commit/ee41d01326ddf48c411c019a4e63343668ebd829


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[miss-islington]

Change by miss-islington :


--
pull_requests: +26875
pull_request: https://github.com/python/cpython/pull/28477

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 7.0 -> 8.0
pull_requests: +26874
pull_request: https://github.com/python/cpython/pull/28476

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Steve Dower]

Steve Dower  added the comment:


New changeset ef9e22b253253615098d22cb49141a2a1024ee3c by Steve Dower in branch 
'main':
bpo-45055: Add retry when downloading externals on Windows (GH-28399)
https://github.com/python/cpython/commit/ef9e22b253253615098d22cb49141a2a1024ee3c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +26813
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28399

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Steve Dower]

Steve Dower  added the comment:

I use git worktree as well (it's great for backporting - I have all the release 
branches checked out all the time), but it likely means that you are regularly 
downloading and extracting these files.

So whatever app is keeping the file handle open is probably doing it reliably 
for one of the files, which is why you'll hit it each time. Once it's been 
copied into place, you won't see it again. And once we have the retry logic in 
the script then you won't see it the first time either.

This should be an easy project, though not trivial to test.

--
keywords: +easy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Guido van Rossum]

Guido van Rossum  added the comment:

I've seen it a few times. I don't do this that often, but since I've discovered 
"git worktree" this has occurred to me a few times (can't recall if it was 
every time, but today wasn't the first). I'm totally happy to be the human in 
the retry loop. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Diego Ramirez]

Change by Diego Ramirez :


--
nosy: +DiddiLeija

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Eryk Sun]

Eryk Sun  added the comment:

I can't directly reproduce the problem. Does it reproduce reliably for you? My 
guess would be that there's a file open in "cpython-source-deps-zlib-1.2.11" 
when get_external.py tries to rename it to "zlib-1.2.11". Maybe an anti-malware 
scanner or content indexer has a file open.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Steve Dower]

Steve Dower  added the comment:

This looks like a typical "lingering open handle" issue, though I can't tell 
whether it's in the destination or the source.

Probably just needs a bit of retry logic (personally I'd be fine putting in 
CPython itself, though that doesn't help this build which is using an 
already-released version). So get_externals.py needs it.

It's certainly not reliably reproducible or the CI system would be on the floor 
;) This runs on every build

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Guido van Rossum]

New submission from Guido van Rossum :

When I make a fresh checkout of the main branch on Windows and type 
"pcbuild\build" it starts downloading some distributions (e.g. sqlite) and then 
starts building. Fine. But at some point there's a whole bunch of errors that 
seem to come from building zlibmodule.c. Re-running pcbuild\build then 
downloads some extra thing and then everything builds to completion.

First set of downloads and selected logs:

Using py -3.9 (found 3.9 with py.exe)
Fetching external libraries...
Fetching bzip2-1.0.6...   
Fetching sqlite-3.35.5.0...
Fetching xz-5.2.2...
Fetching zlib-1.2.11...
Traceback (most recent call last):
  File "C:\Users\gvanrossum\deepfreeze\PCbuild\get_external.py", line 60, in 

main()
  File "C:\Users\gvanrossum\deepfreeze\PCbuild\get_external.py", line 56, in 
main
extract_zip(args.externals_dir, zip_path).replace(final_name)
  File 
"C:\Users\gvanrossum\AppData\Local\Programs\Python\Python39\lib\pathlib.py", 
line 1395, in 
replace
self._accessor.replace(self, target)
PermissionError: [WinError 5] Access is denied: 
'C:\\Users\\gvanrossum\\deepfreeze\\PCbuild\\..\\externals\\cpython-source-deps-zlib-1.2.11'
 -> 'C:\\Users\\gvanrossum\\deepfreeze\\PCbuild\\..\\externals\\zlib-1.2.11'
Fetching external binaries...
Fetching libffi-3.3.0...
Fetching openssl-bin-1.1.1l...
Fetching tcltk-8.6.11.0...
Finished.
Using "C:\Program Files (x86)\Microsoft Visual 
Studio\2019\Enterprise\MSBuild\Current\Bin\msbuild.exe" (found in the Visual 
Studio installation)
Using py -3.9 (found 3.9 with py.exe)

C:\Users\gvanrossum\deepfreeze>"C:\Program Files (x86)\Microsoft Visual 
Studio\2019\Enterprise\MSBuild\Current\Bin\msbuild.exe" 
"C:\Users\gvanrossum\deepfreeze\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m 
/clp:summary /p:Configuration=Release /p:Platform=x64 /p:IncludeExternals=true 
/p:IncludeCTypes=true /p:IncludeSSL=true /p:IncludeTkinter=true 
/p:UseTestMarker= /p:GIT="C:\Program Files\Git\cmd\git.exe"
  Killing any running python.exe instances...
  Regenerate pycore_ast.h pycore_ast_state.h Python-ast.c
  C:\Users\gvanrossum\deepfreeze\Python\Python-ast.c, 
C:\Users\gvanrossum\deepfreeze\Include\inte
  rnal\pycore_ast.h, 
C:\Users\gvanrossum\deepfreeze\Include\internal\pycore_ast_state.h regenerat
  ed.
  Regenerate opcode.h opcode_targets.h
  Include\opcode.h regenerated from Lib\opcode.py
  Jump table written into Python\opcode_targets.h
  Regenerate token-list.inc token.h token.c token.py
  Generated sources are up to date
  Getting build info from "C:\Program Files\Git\cmd\git.exe"
  Building heads/deepfreeze:044e8d866f deepfreeze
  _abc.c

...

Errors:

  Compiling...
  thread.c
  traceback.c
  zlibmodule.c
C:\Users\gvanrossum\deepfreeze\Modules\zlibmodule.c(10,10): fatal error C1083: 
Cannot open includ
e file: 'zlib.h': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc
xproj]
  adler32.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1 
.2.11\adler32.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  compress.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\compress.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.v
cxproj]
  crc32.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\crc32.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vcxp 
roj]
  deflate.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\deflate.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  infback.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\infback.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  inffast.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\inffast.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  inflate.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\inflate.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  inftrees.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\inftrees.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.v 
cxproj]
  trees.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\trees.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vcxp 
roj]
  uncompr.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1