Re: [Distutils] Can't upload sdist: "File already exists"

[Nick Coghlan]

On 6 January 2017 at 07:37, Nick Timkovich  wrote:
> No. My build command was: `python setup.py sdist bdist_wheel` which doesn't
> generate a zip.

We've seen behaviour like this before with the old upload server
implementation, where it was possible to get a particular 5xx error on
upload that meant the server had registered the file as uploaded, but
the upload hadn't actually worked (this bug in the old server was
actually the main reason the default client config suggested on
packaging.python.org was changed).

However, your command output indicates that you're already using the
new upload server, so that's unlikely to have been the problem :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Nick Timkovich]

No. My build command was: `python setup.py sdist bdist_wheel` which doesn't
generate a zip.

On Thu, Jan 5, 2017 at 2:13 PM, Ethan Furman  wrote:

> On 01/05/2017 11:37 AM, Nick Timkovich wrote:
>
> I never determined what was causing my problem, I couldn't reproduce it
>>  on testpypi so I gave up (it's a small package, and if someone wants the
>>  source, they can look at the GH link I have in the metadata).
>>
>
> Do you have both a .zip and a .tar source distribution?
>
>
> --
> ~Ethan~
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Ethan Furman]

On 01/05/2017 11:37 AM, Nick Timkovich wrote:


I never determined what was causing my problem, I couldn't reproduce it
 on testpypi so I gave up (it's a small package, and if someone wants the
 source, they can look at the GH link I have in the metadata).


Do you have both a .zip and a .tar source distribution?

--
~Ethan~
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Nick Timkovich]

I never determined what was causing my problem, I couldn't reproduce it on
testpypi so I gave up (it's a small package, and if someone wants the
source, they can look at the GH link I have in the metadata).

Specifically, why does it say that the file exists, but I *can not see it
anywhere*, either via the web interface or download it with pip --no-binary?

Is it possible that there exists a "huffman-0.1.2.tar.gz" on PyPI from
someone else who may have had the package name, uploaded some releases,
then deleted it? I am 99% confident that I did not publish/delete (all I do
is twine upload dist/*), let alone delete a specific .tar.gz dist (is that
even possible to do from the command-line?). The error happened when I
tried to do a 0.1.1, figured I messed up, then rebuilt/republished with a
bigger number.

The information isn't exposed to me (maybe logs somewhere?), so I'll
probably just accept PyPI is strange like that.

On Thu, Jan 5, 2017 at 12:13 PM, Ethan Furman  wrote:

> On 01/05/2017 09:58 AM, Donald Stufft wrote:
>
> .tar.gz is the recommended format ...
>>
>
> .tar.gz it is!  Thanks!
>
>
> --
> ~Ethan~
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Ethan Furman]

On 01/05/2017 09:58 AM, Donald Stufft wrote:


.tar.gz is the recommended format ...


.tar.gz it is!  Thanks!

--
~Ethan~
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Donald Stufft]

> On Jan 5, 2017, at 12:55 PM, Ethan Furman  wrote:
> 
> On 01/05/2017 08:46 AM, Donald Stufft wrote:
>>> On Jan 5, 2017, at 11:45 AM, Ethan Furman wrote:
> 
>>> Do I need to choose between zip and tar sdists?
>> 
>> Yes.
> 
> I haven't followed Windows O/Ses for a while now -- are they able to easily 
> access tar files?  Or do *nix machines come standard with unzip now?
> 
> Or is it more along the lines of -- you seem to be a developer, so you should 
> be able to install the one you need?  ;)
> 
> --
> ~Ethan~
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


.tar.gz is the recommended format but either one is OK. I think that there is 
still the default tool disparity but getting a tool to open either one is 
fairly trivial, particularly since you can do ``python3 -m tarfile -e 
foobar-1.0.tar.gz. That being said, non-automated downloads account for a tiny 
fraction of the total downloads on PyPI so I wouldn’t worry too much about that 
use case in general.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Ethan Furman]

On 01/05/2017 08:46 AM, Donald Stufft wrote:

On Jan 5, 2017, at 11:45 AM, Ethan Furman wrote:



Do I need to choose between zip and tar sdists?


Yes.


I haven't followed Windows O/Ses for a while now -- are they able to easily 
access tar files?  Or do *nix machines come standard with unzip now?

Or is it more along the lines of -- you seem to be a developer, so you should 
be able to install the one you need?  ;)

--
~Ethan~
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Donald Stufft]

> On Jan 5, 2017, at 11:45 AM, Ethan Furman  wrote:
> 
> Do I need to choose between zip and tar sdists?

Yes.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Ethan Furman]

On 12/22/2016 12:41 PM, Donald Stufft wrote:


There is a fairly new restriction that you can only have *one* sdist per
 release now. That should not apply at all to Wheels and if it is, then it
 is a bug. I can’t reproduce this issue though, so I’m going to guess
 there was some bit of confusion about exact errors here. If someone
 actually cannot upload 1 sdist + N wheels, please leave the state as is
and tell me.


I am also seeing this problem.

My standard files for upload have been a py2 wheel, a py3 wheel, a zip sdist, 
and a tar sdist, and now the tar sdist is being denied as a duplicate.

Do I need to choose between zip and tar sdists?

--
~Ethan~
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Ben Finney]

Nick Timkovich  writes:

> Generally, is it OK (recommended/socially acceptable) to upload
> universal wheels only or is the sdist still nice to have available? I
> guess maybe for pip, but it considering universal, pure-Python wheels
> as "binaries" seems quirky.

None the less, the Wheel format *is* a format intended for installation
(so a “binary” format in that terminology).

A wheel is not suitable for making changes and re-building from source.
So the source distribution still needs to be uploaded as well, to
support that important freedom for the recipient.

-- 
 \   “Liberty, n. One of imagination's most precious possessions.” |
  `\   —Ambrose Bierce, _The Devil's Dictionary_, 1906 |
_o__)  |
Ben Finney

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Donald Stufft]

There is a fairly new restriction that you can only have *one* sdist per 
release now. That should not apply at all to Wheels and if it is, then it is a 
bug. I can’t reproduce this issue though, so I’m going to guess there was some 
bit of confusion about exact errors here. If someone actually cannot upload 1 
sdist + N wheels, please leave the state as is and tell me.

> On Dec 22, 2016, at 3:32 PM, Glyph Lefkowitz  wrote:
> 
>> 
>> On Dec 22, 2016, at 8:54 AM, Ian Cordasco  wrote:
>> 
>> On Thu, Dec 22, 2016 at 9:49 AM, Brett Cannon  wrote:
>>> Because you already uploaded a wheel for version 0.1.2 you can't upload any
>>> other files for that version, else people could accidentally upload e.g. an
>>> sdist with code different from what was already uploaded in the wheel. If
>>> you want an sdist then I would do another release as version 0.1.2post1 with
>>> the wheel and sdist (or whatever the proper post release version format is;
>>> on my phone so a pain to look up right now).
>> 
>> I'm pretty sure that's not correct. Twine is written to specifically
>> upload the wheel first because PyPI will extract metadata from that
>> and display it on the page. It won't do that if the sdist is uploaded
>> first.
>> 
>> I'm not able to reproduce the behaviour Nick is seeing. My only guess
>> is that something changed in Warehouse or the file existed, was
>> deleted, and is now being re-uploaded with the same version. That's
>> not something Warehouse or PyPI allows anymore (republishing with the
>> same version)
> 
> Similarly; I've *definitely* done sdist releases after wheel releases of the 
> same version in the past.
> 
> -glyph
> 
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org 
> 
> https://mail.python.org/mailman/listinfo/distutils-sig 
> 

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Glyph Lefkowitz]

> On Dec 22, 2016, at 8:54 AM, Ian Cordasco  wrote:
> 
> On Thu, Dec 22, 2016 at 9:49 AM, Brett Cannon  wrote:
>> Because you already uploaded a wheel for version 0.1.2 you can't upload any
>> other files for that version, else people could accidentally upload e.g. an
>> sdist with code different from what was already uploaded in the wheel. If
>> you want an sdist then I would do another release as version 0.1.2post1 with
>> the wheel and sdist (or whatever the proper post release version format is;
>> on my phone so a pain to look up right now).
> 
> I'm pretty sure that's not correct. Twine is written to specifically
> upload the wheel first because PyPI will extract metadata from that
> and display it on the page. It won't do that if the sdist is uploaded
> first.
> 
> I'm not able to reproduce the behaviour Nick is seeing. My only guess
> is that something changed in Warehouse or the file existed, was
> deleted, and is now being re-uploaded with the same version. That's
> not something Warehouse or PyPI allows anymore (republishing with the
> same version)

Similarly; I've *definitely* done sdist releases after wheel releases of the 
same version in the past.

-glyph

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Ian Cordasco]

On Thu, Dec 22, 2016 at 9:49 AM, Brett Cannon  wrote:
> Because you already uploaded a wheel for version 0.1.2 you can't upload any
> other files for that version, else people could accidentally upload e.g. an
> sdist with code different from what was already uploaded in the wheel. If
> you want an sdist then I would do another release as version 0.1.2post1 with
> the wheel and sdist (or whatever the proper post release version format is;
> on my phone so a pain to look up right now).

I'm pretty sure that's not correct. Twine is written to specifically
upload the wheel first because PyPI will extract metadata from that
and display it on the page. It won't do that if the sdist is uploaded
first.

I'm not able to reproduce the behaviour Nick is seeing. My only guess
is that something changed in Warehouse or the file existed, was
deleted, and is now being re-uploaded with the same version. That's
not something Warehouse or PyPI allows anymore (republishing with the
same version)
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Nick Timkovich]

So upload the sdist first in the future? What's the difference between
uploading an sdist second with code that differs from what's in the wheel
and vice versa, is PyPI doing some checking?

Generally, is it OK (recommended/socially acceptable) to upload universal
wheels only or is the sdist still nice to have available? I guess maybe for
pip, but it considering universal, pure-Python wheels as "binaries" seems
quirky.

On Thu, Dec 22, 2016 at 10:49 AM, Brett Cannon  wrote:

> Because you already uploaded a wheel for version 0.1.2 you can't upload
> any other files for that version, else people could accidentally upload
> e.g. an sdist with code different from what was already uploaded in the
> wheel. If you want an sdist then I would do another release as version
> 0.1.2post1 with the wheel and sdist (or whatever the proper post release
> version format is; on my phone so a pain to look up right now).
>
> On Wed, Dec 21, 2016, 12:30 Nick Timkovich, 
> wrote:
>
>> I have a little package "huffman" where I build an sdist and wheel
>> (python setup.py sdist bdist_wheel) and both seem to get built and can
>> install fine. I can't seem to upload both to PyPI because the "File already
>> exists":
>>
>> $ twine upload dist/*
>> Uploading distributions to https://upload.pypi.org/legacy/
>> Uploading huffman-0.1.2-py2.py3-none-any.whl
>> Uploading huffman-0.1.2.tar.gz
>> HTTPError: 400 Client Error: File already exists. for url:
>> https://upload.pypi.org/legacy/
>>
>> Subsequent call to upload *just* the tarball fails the same way. I can't
>> see an sdist anywhere, and uploading it via the website or twine just tells
>> me it's already there...somehow. Asking pip to try to give it to me fails
>> though (the binary works, however):
>>
>> $ pip download --no-cache-dir --only-binary :all: huffman==0.1.2
>> Collecting huffman==0.1.2
>>   Downloading huffman-0.1.2-py2.py3-none-any.whl
>>   Saved ./huffman-0.1.2-py2.py3-none-any.whl
>> Successfully downloaded huffman
>>
>>
>> $ pip download --no-cache-dir --no-binary :all: huffman==0.1.2
>> Collecting huffman==0.1.2
>>   Could not find a version that satisfies the requirement huffman==0.1.2
>> (from versions: )
>> No matching distribution found for huffman==0.1.2
>>
>> Am I missing something? I am as sure as I can be that I didn't upload it
>> twice; I bumped my version up one because I figured that may have been it.
>> "twine register" that some guides mention just gets shot down with an HTTP
>> "410 [...] simply upload the file"
>>
>> Cheers,
>> Nick
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can't upload sdist: "File already exists"

[Brett Cannon]

Because you already uploaded a wheel for version 0.1.2 you can't upload any
other files for that version, else people could accidentally upload e.g. an
sdist with code different from what was already uploaded in the wheel. If
you want an sdist then I would do another release as version 0.1.2post1
with the wheel and sdist (or whatever the proper post release version
format is; on my phone so a pain to look up right now).

On Wed, Dec 21, 2016, 12:30 Nick Timkovich,  wrote:

> I have a little package "huffman" where I build an sdist and wheel (python
> setup.py sdist bdist_wheel) and both seem to get built and can install
> fine. I can't seem to upload both to PyPI because the "File already exists":
>
> $ twine upload dist/*
> Uploading distributions to https://upload.pypi.org/legacy/
> Uploading huffman-0.1.2-py2.py3-none-any.whl
> Uploading huffman-0.1.2.tar.gz
> HTTPError: 400 Client Error: File already exists. for url:
> https://upload.pypi.org/legacy/
>
> Subsequent call to upload *just* the tarball fails the same way. I can't
> see an sdist anywhere, and uploading it via the website or twine just tells
> me it's already there...somehow. Asking pip to try to give it to me fails
> though (the binary works, however):
>
> $ pip download --no-cache-dir --only-binary :all: huffman==0.1.2
> Collecting huffman==0.1.2
>   Downloading huffman-0.1.2-py2.py3-none-any.whl
>   Saved ./huffman-0.1.2-py2.py3-none-any.whl
> Successfully downloaded huffman
>
>
> $ pip download --no-cache-dir --no-binary :all: huffman==0.1.2
> Collecting huffman==0.1.2
>   Could not find a version that satisfies the requirement huffman==0.1.2
> (from versions: )
> No matching distribution found for huffman==0.1.2
>
> Am I missing something? I am as sure as I can be that I didn't upload it
> twice; I bumped my version up one because I figured that may have been it.
> "twine register" that some guides mention just gets shot down with an HTTP
> "410 [...] simply upload the file"
>
> Cheers,
> Nick
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Can't upload sdist: "File already exists"

[Nick Timkovich]

I have a little package "huffman" where I build an sdist and wheel (python
setup.py sdist bdist_wheel) and both seem to get built and can install
fine. I can't seem to upload both to PyPI because the "File already exists":

$ twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Uploading huffman-0.1.2-py2.py3-none-any.whl
Uploading huffman-0.1.2.tar.gz
HTTPError: 400 Client Error: File already exists. for url:
https://upload.pypi.org/legacy/

Subsequent call to upload *just* the tarball fails the same way. I can't
see an sdist anywhere, and uploading it via the website or twine just tells
me it's already there...somehow. Asking pip to try to give it to me fails
though (the binary works, however):

$ pip download --no-cache-dir --only-binary :all: huffman==0.1.2
Collecting huffman==0.1.2
  Downloading huffman-0.1.2-py2.py3-none-any.whl
  Saved ./huffman-0.1.2-py2.py3-none-any.whl
Successfully downloaded huffman


$ pip download --no-cache-dir --no-binary :all: huffman==0.1.2
Collecting huffman==0.1.2
  Could not find a version that satisfies the requirement huffman==0.1.2
(from versions: )
No matching distribution found for huffman==0.1.2

Am I missing something? I am as sure as I can be that I didn't upload it
twice; I bumped my version up one because I figured that may have been it.
"twine register" that some guides mention just gets shot down with an HTTP
"410 [...] simply upload the file"

Cheers,
Nick
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig