[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily  added the comment:

I've updated and synced the What's New files across master, 3.7, 3.6, 3.5 (PR 
pending RM merge), and 2.7.  So I think we are finally done here.

--
priority: deferred blocker -> normal
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily  added the comment:


New changeset 4fde701133643d09354ec4bda6bdf78d85aed797 by Ned Deily in branch 
'2.7':
bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7624)
https://github.com/python/cpython/commit/4fde701133643d09354ec4bda6bdf78d85aed797


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +7243

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +7242

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily  added the comment:


New changeset 1b5731e2761a9a1b7394b3a81ed267f94fde42ca by Ned Deily in branch 
'3.6':
bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7622)
https://github.com/python/cpython/commit/1b5731e2761a9a1b7394b3a81ed267f94fde42ca


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +7241

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily  added the comment:


New changeset 144493dd7f2640f7e9091862ece3e0a6aca07884 by Ned Deily (Miss 
Islington (bot)) in branch '3.7':
bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7620) (GH-7621)
https://github.com/python/cpython/commit/144493dd7f2640f7e9091862ece3e0a6aca07884


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[miss-islington]

Change by miss-islington :


--
pull_requests: +7240

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily  added the comment:


New changeset 9d6171ded5c56679bc295bacffc718472bcb706b by Ned Deily in branch 
'master':
bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7620)
https://github.com/python/cpython/commit/9d6171ded5c56679bc295bacffc718472bcb706b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +7239
stage: backport needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan  added the comment:

No, I'm not currently working on this (to be completely honest, I entirely 
forgot we hadn't backported the What's New updates to the various branches).

They'll need to be independent updates, since they affect a different file for 
each release (although they can then be cherry-picked forward into the 
development branch)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily  added the comment:

Nick, were you planning to do the work for the other (non-3.6) branches?  If 
not, can someone else pick this up please?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily added the comment:

There still might be time to get this into 3.5.4 final, if Larry agrees.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan added the comment:

Moving to deferred blocker and taking 3.6 off the affected versions list, since 
the 3.6 What's New has now been updated appropriately.

Items still to be done:

- make a similar change to the 3.5 What's New on the 3.5 branch
- update the maintenance release changes section in the 2.7 What's New on the 
2.7 branch
- cherry-pick the 2.7, 3.5, & 3.6 updates in order to make a combined PR for 
the development branch

--
priority: release blocker -> deferred blocker
stage: needs patch -> backport needed
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan added the comment:


New changeset e1b690370fd8f93bef1e69eeea2695f95a7cfff5 by Nick Coghlan in 
branch '3.6':
bpo-23404: `make regen-all` What's New entry (#2128)
https://github.com/python/cpython/commit/e1b690370fd8f93bef1e69eeea2695f95a7cfff5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan added the comment:

Thanks for that Victor. I started to review the PR, and then realised my 
suggestions were going to be extensive enough that it made more sense to post 
an alternate PR for you to review: https://github.com/python/cpython/pull/2128

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +2182

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

I wrote https://github.com/python/cpython/pull/2127 to document "make 
regen-all". I'm not sure that it's the correct way to document such build 
change in minor 3.6 releases. Can someone please take a look?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2181

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily added the comment:

As Nick noted, this change needs to be properly documented for 3.6.2 and other 
maintenance releases.

--
nosy: +benjamin.peterson, larry
priority: critical -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan added the comment:

As noted on python-dev, it would be helpful if this change was mentioned in the 
2.7, 3.5, and 3.6 What's New documents under a "Notable changes in maintenance 
releases" section.

For 2.7, that would be a retitling of the existing section "New features added 
to Python 2.7 maintenance releases": 
https://docs.python.org/dev/whatsnew/2.7.html#new-features-added-to-python-2-7-maintenance-releases

For 3.5 and 3.6, this would be a new entry at the end of the document, with 
"Change to handling of pre-generated files" as the sole subsection.

(As another example of an existing section along these lines, see 
https://docs.python.org/dev/whatsnew/3.4.html#changed-in-3-4-3)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

I modified Python 2.7, 3.5, 3.6 and master (3.7). I prefer to keep the issue 
open at least one week in a regression is found, like issue #30273 (this one 
should already be fixed).

Maybe the devguide should be updated too?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:


New changeset 8a19eb24c97ef43e9fc7d45af180334ac8093545 by Victor Stinner in 
branch '2.7':
bpo-23404: make touch becomes make regen-all (#1466)
https://github.com/python/cpython/commit/8a19eb24c97ef43e9fc7d45af180334ac8093545


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1565

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:


New changeset ab6b962ef241be97536573d7490ce1cfc74fde18 by Victor Stinner in 
branch '3.5':
bpo-23404: make touch becomes make regen-all (#1405) (#1461) (#1465)
https://github.com/python/cpython/commit/ab6b962ef241be97536573d7490ce1cfc74fde18


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1563

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:


New changeset 9d02f562961efd12d3c8317a10916db7f77330cc by Victor Stinner in 
branch '3.6':
[3.6] bpo-23404: make touch becomes make regen-all (#1405) (#1461)
https://github.com/python/cpython/commit/9d02f562961efd12d3c8317a10916db7f77330cc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:


New changeset b109a1d3360fc4bb87b9887264e3634632d392ca by Victor Stinner in 
branch 'master':
bpo-30273: Update sysconfig (#1464)
https://github.com/python/cpython/commit/b109a1d3360fc4bb87b9887264e3634632d392ca


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1562

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

I started a thread on python-dev to ask if it's ok to backport the change to 
2.7, 3.5 and 3.6.

I started to backport the change to 3.6:
https://github.com/python/cpython/pull/1461

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1557

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

I just created the issue #30259: Test somehow that generated files are up to 
date: run make regen-all.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:


New changeset a5c62a8e9f0de6c4133825a5710984a3cd5e102b by Victor Stinner in 
branch 'master':
bpo-23404: make touch becomes make regen-all (#1405)
https://github.com/python/cpython/commit/a5c62a8e9f0de6c4133825a5710984a3cd5e102b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

> If we go in this direction, maybe we can go further and also run autoconf && 
> autoheader?

Oh, I just noticed that there is already a "make autoconf" target, I didn't 
know.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

Nick Coghlan: "Could we use "regen-*" as the pattern for the file regeneration 
targets?"

Sure, done.

I checked the CPython documentation, build scripts, etc. I confirm that 
"rebuild" usually means "recompile Python from scratch", it's even an explicit 
action in the Windows build system.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan added the comment:

Could we use "regen-*" as the pattern for the file regeneration targets?

I'd expect "make rebuild-all" to be akin to "make clean && make", since in the 
context of a C project, "build" maps to "generate the output binaries" for me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

https://github.com/python/cpython/pull/1405#pullrequestreview-35923931

Zachary Ware: "LGTM, but I haven't fully parsed how the old mess worked. We 
should also have CI (either Travis or buildbot) confirming that make 
rebuild-all doesn't leave any checked-in files modified."

Yeah, such CI would be useful.

--

If we go in this direction, maybe we can go further and also run autoconf && 
autoheader?

While fixing bpo-30232, I had to do a second commit 
(9ed34a89532763cf89f5e11fffb91ef7dee29fed) because my first one 
(5facdbb29169c2799c42f887cef4cd9d087b0167) only modified configure.ac. Well, it 
might be a bot on pull requests, but an extra check wouldn't hurt anyway ;-)

A problem with autoconf is that the result depends on the exact autoconf 
version :-/ For example, autoconf 2.69 of my Fedora 25 wants to remove 
runstatedir from configure. I don't know what is runstatedir, so I had to 
revert these autoconf changes on configure before pushing...

haypo@selma$ touch configure.ac && autoconf && git diff

diff --git a/configure b/configure
index a20cf97..a6b13e8 100755
--- a/configure
+++ b/configure
@@ -783,7 +783,6 @@ infodir
 docdir
 oldincludedir
 includedir
-runstatedir
 localstatedir
 sharedstatedir
 sysconfdir
(...)

A solution would be to use a fixed autoconf version and ensure that configure 
is always generated with the same autoconf version. Maybe Gentoo is a good OS 
for such CI such Gentoo provides multiple versions of autotools, whereas other 
Linux distro usually only provide one version of all autotools tools.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

The issue #23404 has been marked as a duplicate of this bug: "AMD64 FreeBSD 9.x 
3.x" tries to rebuild Include/opcode.h, timestamp issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

I rewrote Python (UNIX/BSD) build system to not rebuild generated files based 
on file modification time: the action is now explicit. This change should not 
only fix buildbots, but also ease cross-compilation, and more generally make 
Python build less painful! No more funny error when the system only provides 
Python 2.6 and so "make touch" doesn't work, whereas "make touch" was the 
obvious workaround to avoid trying to regenerated files.

The change:
https://github.com/python/cpython/pull/1405

My change adds a global "rebuild-all", but this command is made of multiple 
subcommands:

  - rebuild-ast: Include/Python-ast.h and Python/Python-ast.c
  - rebuild-grammar: Include/graminit.h and Python/graminit.c
  - rebuild-importlib: Python/importlib_external.h and Python/importlib.h
  - rebuild-opcode: Include/opcode.h
  - rebuild-opcode-targets: Python/opcode_targets.h
  - rebuild-typeslots: Objects/typeslots.inc

rebuild-all is not needed if you only want to update AST for example: just run 
"make rebuild-ast && make".

--

As you may expect, the change is quite big for a very sensitive part of Python: 
the build system. Do we want to backport such major change?

Another solution for stable branches to to remove the rebuild of generated 
files from configure and Makefile... Just hope that we will not need it 
anymore? Not sure if it's a good idea. I expect fixes in importlib, so 
"rebuild-importlib" will be needed at least.

Even if I don't trust my own work(!), I would suggest to review carefully my 
change, test it on all buildbots, play with it, and then backport it to 2.7, 
3.5 and 3.6.

Note: Windows is not affected by this bug, right?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1513

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Martin Panter added the comment:

A while ago I wrote a patch targetting Issue 22359 that may be a starting point 
for “make regen”: . It 
pulled out three recipes into separate “phony” targets: “make graminit 
importlib importlib_external”.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan added the comment:

As Martin notes, my concern is that I can go for years without having to touch 
the low level files (since syntax changes are rare), so it's important to me to 
make it easy to remember and/or relearn how to regenerate them.

However, suitable make commands (e.g. a "make regen" target, with "make 
regen-X" subtargets) would qualify as "easy to remember", and would also be 
amenable to automated testing in combination with a `make clobber-gen` command.

(Tangent on target naming: I think 'rebuild' would be confusing, while 
'regenerate' is a bit long, especially if there are more specific subtargets. 
'regen' is just a shortened version of the latter)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Brett Cannon]

Brett Cannon added the comment:

Yep, that's exactly what I was suggesting, Martin (and sorry if I missed you 
saying that earlier; on vacation so trying to keep email/open source time to 
minimum :) .

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Martin Panter added the comment:

Do you mean a separate makefile rule to rebuild the generated files, rather 
than rebuilding them in a normal “make all” build? I would support this; this 
is what I meant with my “make regenerate” proposal in response to Nick linked 
above.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Brett Cannon]

Brett Cannon added the comment:

Would it make sense to have a `make rebuild` or something which explicitly 
ignores prebuilt files? That could then also serve as non-obvious documentation 
on how to build those files in the first place.

--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Kubilay Kocak]

Changes by Kubilay Kocak :


--
nosy: +koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Martin Panter added the comment:

Last time I proposed removing the automatic rebuilding of checked-in generated 
files, it seemed getting a consensus would not be trivial. Nick seemed strongly 
against changing the status quo: 
.
 He did say it has been too hard to remember how to rebuild generated files in 
the past.

Documenting the quirks of the build system would be good even if we keep the 
automatic regeneration.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

> Anyway my preference was to stop automatically regenerating files, and to 
> keep the build system as simple as practical without hacks like the BOOT="#" 
> flag.

In the past, this issue bite me so many times on Solaris, FreeBSD, etc. One 
because there was not "python" command, then because "python" was an old python 
2.6 and so hg touch didn't work, etc.

It's "nice" to try to keep track of build dependencies, but in pratice it is 
super painful. So I'm also in favor of removing these build dependencies based 
on file modification file, and maybe document how to force a rebuild (do you 
really need to do that? developers hacking special files know how to rebuild 
generated files, no?).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[STINNER Victor]

STINNER Victor added the comment:

FYI 3 FreeBSD buildbots are currently broken because of this issue.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Martin Panter added the comment:

I had mainly been using my boot-flag.patch with BSD Make (bmake) rather than 
Gnu Make. It seems I was relying on a bug in BSD Make that merges escaped 
newlines in command lines, despite Posix and Gnu Make. Anyway, Chi’s fix seems 
appropriate.

I am posting the patches I used for 3.5 and 2.7, in case others are interested. 
The 3.5 one would also need Chi’s fix merging the “_freeze_importlib” line.

It may be possible to have the configure script adjust the default for the BOOT 
flag rather than setting GENERATED_COMMENT. But the side effect would be 
disabling regeneration of additional things (e.g. that aren’t specifically 
broken with cross compilation.

Anyway my preference was to stop automatically regenerating files, and to keep 
the build system as simple as practical without hacks like the BOOT="#" flag.

--
Added file: http://bugs.python.org/file46772/boot-flag.py3.5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Changes by Martin Panter :


Added file: http://bugs.python.org/file46773/boot-flag.py2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Zachary Ware]

Zachary Ware added the comment:

I've disabled the Touch step on the buildbots until this is fixed.

--
keywords: +buildbot
nosy: +zach.ware
priority: normal -> critical
versions: +Python 2.7, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Chi Hsuan Yen]

Chi Hsuan Yen added the comment:

Just tried boot-flag.patch with 
https://github.com/python/cpython/commit/16e07881bd3867d471dd0a25c5117672b65c90ee.
 Doesn't work out-of-box:

#./Programs/_freeze_importlib \
./Lib/importlib/_bootstrap.py Python/importlib.h
/bin/sh: line 1: ./Lib/importlib/_bootstrap.py: Permission denied
make: *** [Makefile:749: Python/importlib.h] Error 126

My fix is: merge the two lines into a single one. See my attached patch.

By the way, the BOOT hack is quite similar to GENERATED_COMMENT in introduced 
in issue27641. Maybe they can merge?

My environment:

* Arch Linux x86_64
* /bin/sh symlinks to bash, which is 4.4.12
* GNU Make 4.2.1

--
nosy: +Chi Hsuan Yen
Added file: http://bugs.python.org/file46620/boot-flag.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Martin Panter added the comment:

Okay so my “make -t” trick has various flaws. You still have to know the 
filenames to specify, it requires Makefile to be generated by configure in the 
source tree, and it creates empty files if you use it in a separate build 
directory.

Another idea: instead of the “make touch” recipe, we add a simple shell script. 
Call it say “touch-bootstrap.sh”. Like what “make touch” already does in Python 
2, but without embedding it as a makefile rule.

Or add a flag variable so you can do a build without running any of the 
regeneration rules or worrying about timestamps: make BOOT="#".

--
Added file: http://bugs.python.org/file44627/boot-flag.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Martin Panter added the comment:

Perhaps you can use “make -t ” instead of “make touch”. If you know the 
filename you want to avoid regenerating, that would not need any extra tools, 
no Mercurial or Git or any other version control. E.g.:

make -t Include/Python-ast.h

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ezio Melotti]

Changes by Ezio Melotti :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Berker Peksag]

Berker Peksag added the comment:

Patch LGTM. Thanks Carol.

I will left this issue open in case someone wants to work on a git equivalent 
of make touch as described by Ned in msg235514.

--
components:  -Devguide
nosy: +berker.peksag
stage: patch review - needs patch
type:  - enhancement
versions: +Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23404
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Carol Willing]

Carol Willing added the comment:

Using Ned Deily's feedback, I have added a devguide patch to clarify that 'make 
touch' only works with a Mercurial clone and was not designed for git clones. 
This patch, iss23404devguide.patch, resolves the devguide portion of this issue 
unless anyone has further wording changes.

--
nosy: +willingc
stage: needs patch - patch review
Added file: http://bugs.python.org/file40044/iss23404devguide.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23404
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Vinson Lee]

Vinson Lee added the comment:

With the supplied patch on CentOS 6 , make passes and make test passes with 
the exception of test_readline that is Issue19884.

372 tests OK.
1 test failed:
test_readline
7 tests altered the execution environment:
test_calendar test_distutils test_float test_locale test_strptime
test_types test_warnings
11 tests skipped:
test_devpoll test_gdb test_kqueue test_msilib test_ossaudiodev
test_startfile test_tk test_ttk_guionly test_winreg test_winsound
test_zipfile64
Re-running failed tests in verbose mode
Re-running test 'test_readline' in verbose mode
testHistoryUpdates (test.test_readline.TestHistoryManipulation) ... ok
test_write_read_append (test.test_readline.TestHistoryManipulation) ... ok
test_init (test.test_readline.TestReadline) ... FAIL

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23404
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Vinson Lee]

Vinson Lee added the comment:

I tried a hg copy of the repository. make touch worked for me.

./configure
make touch
make

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23404
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Ned Deily]

Ned Deily added the comment:

Unfortunately, those syntax differences aren't the only problems you could run 
into so making these syntax changes isn't really a general solution, IMO. 'make 
touch' is the documented and supported way to ensure that the unnecessary build 
steps are not run.  But, as you discovered, 'make touch' only works with an hg 
copy of the repo; it was not designed for git clones.  To fully support git 
clones, there would need to be a git equivalent of the hg scripts called by 
'make touch'.  This would be a worthwhile project for someone interested in git 
support to work on and would be considered for inclusion in the repo.

--
components: +Devguide
nosy: +ezio.melotti
resolution: not a bug - 
stage: resolved - needs patch
title: Python 3.5 does not build with Python 2.6. - 'make touch' does not work 
with git clones of the source repository
type: compile error - 

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23404
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com