[issue11104] distutils sdist ignores MANIFEST
John Dennis jden...@redhat.com added the comment: The changesets are not in the release27-maint branch. sdist still does not generate a correct archive for release, this is a very serious regression. -- resolution: fixed - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: The changesets are not in the release27-maint branch. Where did you look? This looks like a Subversion branch name, but now we’re using Mercurial. If you look a few messages up, you’ll see that a changeset was committed to the 2.7 branch and will be included in 2.7.3. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment: Yep - 2.7.2 was released 11th June 2011, the fix was committed Aug 1st 2011. So it won't be in the current 2.7 release. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: Obviously I can’t fix past releases. Sometimes the time machine is not available :) -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 5d3e22d69d4f by Éric Araujo in branch '3.2': Fix regression with distutils MANIFEST handing (#11104, #8688). http://hg.python.org/cpython/rev/5d3e22d69d4f -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 21feea7f35e5 by Éric Araujo in branch '2.7': Fix regression with distutils MANIFEST handing (#11104, #8688). http://hg.python.org/cpython/rev/21feea7f35e5 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: I did some more work on the patch and committed. Thanks again to both of you. I also thought again about this remark: Changeset r83996 was introduced to prevent sdist from overwriting a project maintained manifest by testing for a comment at the head of the manifest file. It's not clear to me this was necessary because the write_manifest() should never have been called if the template was absent but a manifest existed. I also wonder if that was necessary; I did not fully understand the bug at the time. Unfortunately, now that we have a handful of releases out there with this change, we have to support the magic comment. For distutils2 however (a.k.a. “packaging” module in 3.3), I’ll open another bug report and if the same bugs apply (MANIFEST handling is a bit different there), I’ll try to fix the bug without using a magic comment. -- components: -Distutils2 resolution: - fixed stage: commit review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment: Éric mentioned that i should check that this behaviour matches the documentation. I have gone and looked for all instances of MANIFEST in the documentation and found one place which was inconsistent. I've added the doc patch to the patch. Please review this new version. -- Added file: http://bugs.python.org/file22463/manifest-respect-3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: Thanks for your work. I will make a final test and review and commit it in the coming days. I’ll adapt the versionchanged text, since I won’t commit this in 3.1 (in security mode now, not bugfix mode). -- priority: normal - high ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment: I have 2 patches, with tests, that applies on python2.7 and the python3 series of branches, attached this ticket. I have also got a signed contributor agreement lodged with the PSF. Can I please have someone either apply my patches or tell me what I need to do in order to change them if they are being rejected. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: I have outstanding comments and questions on the review page (follow the review link on the right of the 3.x patch). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment: Oh! I didn't see any notification that there was a review done. Thanks, I'll attend to that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment: This patch is an updated patch that fixes the things noted in the review from eric.araujo. -- Added file: http://bugs.python.org/file22437/manifest-respect-3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment:
Updated the patch to address the 'why not use .strip()' question. I used
.rstrip('\r\n') on the basis that filenames may have leading or trailing
spaces, and if you need that, you need to be able to specify that in a
MANIFEST, but it is perfectly logical to disallow them, so here's a patch that
doesn't support them.
It also reduces the line count by 2 because I'm composing the 'comment' and
'blank line' cases.
--
Added file: http://bugs.python.org/file22449/manifest-respect-3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11104
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: Thanks for your work, the code can now be fixed; I’ll have time to commit in a few days. Can you check if the documentation still accurately describes the behavior? (search for MANIFEST in Doc/distutils/*) -- stage: - commit review versions: +Python 3.3 -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment: I've taken the sdist.patch and wrote some tests for it. The resulting patch is attached as 'manifest-respect.patch'. -- nosy: +jerub Added file: http://bugs.python.org/file22242/manifest-respect.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Stephen Thorne step...@thorne.id.au added the comment: This patch is tested against the 3.1 and default branches, the previous patch attached was against the 2.7 branch. -- Added file: http://bugs.python.org/file22243/manifest-respect-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Changes by John Dennis jden...@redhat.com: -- nosy: +dmalcolm ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
John Dennis jden...@redhat.com added the comment: No, I don't think I'm going to turn the tarball into a unit test, etc. I didn't break the code but I did report the problem, researched the history, provided a clear explanation, a reproducer and a patch to fix the problem. I think I've done my part as a community member. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: I didn’t mean to imply you had to, I just asked if you wanted to. Reporting the bug is a valuable contribution indeed, I’ll take over for the rest. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: This behavior change is actually present in 2.7.1, 3.1.3 and 3.2. I’m concerned that you’ve found a bug with it (thanks for the report), and that we may have to change behavior again. First, could you open another report about comment handling in read_manifest? I overlooked that, and it should be fixed as quickly as possible, although there’s very little change this will make it into 3.2.0 (we’ll have 3.2.1 though). Second, can you give us a test script (shell or Python) or a set of instructions we can reproduce to observe the behavior you say is defective? Thanks in advance. -- components: +Distutils2 versions: +3rd party, Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
John Dennis jden...@redhat.com added the comment: $ tar xzf distutils_bug.tar.gz $ cd distutils_bug $ ./setup.py sdist $ ./setup.py sdist running sdist running check warning: sdist: manifest template 'MANIFEST.in' does not exist (using default file list) not writing to manually maintained manifest file 'MANIFEST' creating foobar-1.0 making hard links in foobar-1.0... hard linking README - foobar-1.0 hard linking setup.py - foobar-1.0 creating dist Creating tar archive removing 'foobar-1.0' (and everything under it) $ cat MANIFEST README setup.py my_special_file Note, the MANIFEST should have been read and the resulting tar file should have had my_special_file in it. sdist complains about a missing MANIFEST.in template which it shouldn't, it should use the existing MANIFEST, it also emits a warning about not overwriting a manually maintained MANIFEST which it shouldn't. It should just use the existing MANIFEST. -- Added file: http://bugs.python.org/file20672/distutils_bug.tar.gz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Éric Araujo mer...@netwok.org added the comment: Thank you, the bug is clear now. To fix this regression, the first step is to turn your tarball and instructions into a unit test and then fix the logic in the code. If you want to do it, there are some process guidelines at http://wiki.python.org/moin/Distutils/FixingBugs -- assignee: tarek - eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
New submission from John Dennis jden...@redhat.com: The behaviour of sdist has changed dramatically in Python 2.7. Some projects prefer to maintain their own manifest file instead of utilizing automatic manifest generation from a template. The defined behaviour of sdist is to check for the presence of both a template and a manifest, if the template is absent but a manifest exists sdist is supposed to read the existing manifest, it no longer does this. Instead it creates a default file list with the catastrophic result of omitting the bulk of a projects files. It appears this bug was introduced in r81255 and is discussed in #8688. Unfortunately #8688 contained a number of different issues and was closed addressing only a subset of the problems. Changeset r83996 was introduced to prevent sdist from overwriting a project maintained manifest by testing for a comment at the head of the manifest file. It's not clear to me this was necessary because the write_manifest() should never have been called if the template was absent but a manifest existed. Even after the application of changeset r83996 one of the fundamental problems in #8688 remained, the manifest is not read. The solution is to check for both the manifest and template (as was formerly done) and if the template is absent but the manifest exists then the manifest should be read. I have made modifications to get_file_list() to reintroduce the defined behaviour. With the introduction of r83996 it is now legal syntax to have comments in the manifest however read_manifest() was not enhanced to account for the possible presence of comments. I also modified read_manifest() to handle comments. These suggested fixes are attached as a patch against the current 2.7 maintenance branch. I've also attached a file with the two modified methods because sometimes it's difficult to comprehend a patch. -- assignee: tarek components: Distutils messages: 12 nosy: eric.araujo, jdennis, tarek priority: normal severity: normal status: open title: distutils sdist ignores MANIFEST type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Changes by John Dennis jden...@redhat.com: -- keywords: +patch Added file: http://bugs.python.org/file20655/sdist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Changes by John Dennis jden...@redhat.com: Added file: http://bugs.python.org/file20656/sdist_new ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Changes by John Dennis jden...@redhat.com: Removed file: http://bugs.python.org/file20655/sdist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Changes by John Dennis jden...@redhat.com: Removed file: http://bugs.python.org/file20656/sdist_new ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Changes by John Dennis jden...@redhat.com: Added file: http://bugs.python.org/file20658/sdist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11104] distutils sdist ignores MANIFEST
Changes by John Dennis jden...@redhat.com: Added file: http://bugs.python.org/file20659/sdist_new ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11104 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
