Re: [Python-Dev] PEP 514 and pywin32

2016-12-28 Thread Burkhard Meier 
Try the 'gohlke' download site. Whenever getting stuck in some of the
newest Python 3.x versions, that side usually has installers that work. It
did work for me just now, using Python 3.6.0 64-bit on Windows 10 64-bit OS.

[image: Inline image 1]

Burkhard

On Tue, Dec 27, 2016 at 9:17 PM, Glenn Linderman  wrote:

> So today I tried to install pywin32 on my new Python 3.6.0 and got the
> following error:
>
> ---
> Cannot install
> ---
> Python version 3.6-32 required, which was not found in the registry.
> ---
> OK
> ---
>
> Seems like pywin32, although built for 3.6, doesn't understand or conform
> to the PEP 514? So the installer doesn't work? I suspect maybe the code
> would still work, if it would install. I also noted that pip cannot find a
> compatible pywin32, and PyPI only reports compatibility through Python 3.3.
>
> 1. Where should this be reported? SourceForge?
> 2. Anyone know a workaround?
>
> --
> Glenn
> --
> Experience is that marvelous thing that enables you to recognize a
> mistake when you make it again. -- Franklin Jones
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> burkhardameier%40gmail.com
>
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for feedback about fixing `ftplib' (issue25458)

2016-12-28 Thread Ivan Pozdeev via Python-Dev 
Re: [Python-Dev] Asking for feedback about fixing `ftplib' (issue25458) 
On 25.12.2016 17:21, Giampaolo Rodola' wrote:
From what I understand the problem should be fixed in urllib so that 
it always closes the FTP connection. I understand this is what happens 
in recent 3.x versions but not on 2.7.
urllib in 2.7 should indeed be fixed to close FTP connections rather 
than leave them dangling, no question about that.
(ftpcache probably has to go in all branches as a result since it's now 
useless - but that's another issue entirely)


The question is, in both 2.x and 3.x, urllib currently doesn't close the 
control connection gracefully. It just closes the socket immediately 
after reading all data on the data connection - without issuing QUIT or 
even reading the end-of-transfer response.

By doing this, it gets away with not calling voidresp().
(RFC 939 specifies that the server should execute the equivalent of ABOR 
and QUIT upon unexpected close of control connection.)
As for ftplib, I don't like the solution of using a 
`transfer_in_progress` flag in all commands. I see that as a cheap way 
to work around the fact that the user may forget to call voidresp() 
and close the socket after nt/transfercmd(). It probably makes sense 
to just update nt/transfercmd() doc instead and make that clear.
I tried to fix urllib so that it's guaranteed to call voidresp(), and 
failed. Too complicated, effectively reimplementing a sizable part of 
FTP client logic is required, perhaps even monkey-patching ftplib to be 
able to set flags in the right places. That's why I thought that just 
requiring the user to call it themselves and calling it a day would be 
asking too much from users and the easy way out - ftplib currently dumps 
too much work on its users that it ought to do itself instead.


The fact that in FTP, one cannot send another command before the 
previous one is complete (or rather, one can, but the server won't 
respond to it until then) means that FTP is inherently stateful. So, 
ftplib, to be a usable library, needs to either encapsulate this 
statefulness, or provide building blocks with all the stock logic parts 
so that only truly application-specific logic has to be added to make a 
robust solution.


With simple commands (|voidcmd|) and self-contained transfer commands 
(retrlines/retrbinary), ftplib does incapsulate statefulness, by 
handling them atomically. But with transfercmd(), it returns control to 
the user halfway through.
At this point, if ftplib doesn't itself keep track that the control 
connection is currently in invalid state for the next command, the user 
is forced to do that themselves instead. And guess what - urllib has to 
use ftplib through a wrapper class that does exactly that. That's 
definitely a "stock logic part" 'cuz it's an integral part of FTP rather 
than something specific to user logic.


The other "stock logic part" currently missing is error handling during 
transfer. If the error's cause is local (like a local exception that 
results in closing the data socket prematurely, or the user closing it 
by hand, or an ABOR they sent midway), the error code from the 
end-of-transfer response should be ignored, otherwise, it shouldn't.
Nothing currently provides this logic. ftplib.retr*() never ignore the 
code - because they never abort the transfer - but they don't handle 
exceptions, either, so they wouldn't read the response if one is raised 
on data socket read or in the callback. urllib used to handle the 
response in an overridden socket close handler, and it was forced to 
always ignore the code because it's impossible to check from there if 
there was an exception or if the data socket was closed prematurely.

--
Other than making ftplib keep track of the session's internal state 
while the user has control and providing the custom error handling logic 
to them, another solution is to never release control to the user 
midway, i.e. ban transfercmd() entirely and only provide self-contained 
retr*()/dir()/whatever, but let the callback signal them to abort the 
transfer. That way, the state would be kept implicitly - in the 
execution pointer.




--
Giampaolo - http://grodola.blogspot.com



--
Regards,
Ivan

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 514 and pywin32

2016-12-28 Thread Steve Dower 
It's likely that they're using the broken version of bdist_wininst for their 
installer (I thought Mark reported the issue and had a workaround though...). 
It's already fixed, but hasn't been released yet.

Another workaround is to use "wheel convert" on the exe and then install the 
wheel. You miss out on their post-install steps, but most people don't need 
those anyway.

Cheers,
Steve

Top-posted from my Windows Phone

-Original Message-
From: "Burkhard Meier" 
Sent: ‎12/‎28/‎2016 1:43
To: "Glenn Linderman" 
Cc: "Python Dev" 
Subject: Re: [Python-Dev] PEP 514 and pywin32

Try the 'gohlke' download site. Whenever getting stuck in some of the newest 
Python 3.x versions, that side usually has installers that work. It did work 
for me just now, using Python 3.6.0 64-bit on Windows 10 64-bit OS.






Burkhard


On Tue, Dec 27, 2016 at 9:17 PM, Glenn Linderman  wrote:

So today I tried to install pywin32 on my new Python 3.6.0 and got the 
following error:

---
Cannot install
---
Python version 3.6-32 required, which was not found in the registry.
---
OK   
---

Seems like pywin32, although built for 3.6, doesn't understand or conform to 
the PEP 514? So the installer doesn't work? I suspect maybe the code would 
still work, if it would install. I also noted that pip cannot find a compatible 
pywin32, and PyPI only reports compatibility through Python 3.3.

1. Where should this be reported? SourceForge?
2. Anyone know a workaround?


-- 
Glenn 


Experience is that marvelous thing that enables you to recognize a
mistake when you make it again. -- Franklin Jones 

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/burkhardameier%40gmail.com___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merry Christmas to me, and Python users everywhere

2016-12-28 Thread Barry Warsaw 
On Dec 27, 2016, at 01:34 PM, Glenn Linderman wrote:

>Thanks for this extra information, Ethan. That points at Gmail pretty
>conclusively as the source of the reduction in number of messages. Since it
>has long been known that Gmail suppresses CC or BCC to self, it is likely
>that suppressing duplicate messages from cross-posted mailing lists is also
>done... likely achieved due to matching Message-Id values.

https://wiki.list.org/x/4030680

Cheers,
-Barry
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for feedback about fixing `ftplib' (issue25458)

2016-12-28 Thread Giampaolo Rodola' 
On Wed, Dec 28, 2016 at 3:57 PM, Ivan Pozdeev  wrote:

> On 25.12.2016 17:21, Giampaolo Rodola' wrote:
>
> From what I understand the problem should be fixed in urllib so that it
> always closes the FTP connection. I understand this is what happens in
> recent 3.x versions but not on 2.7.
>
> urllib in 2.7 should indeed be fixed to close FTP connections rather than
> leave them dangling, no question about that.
>

To me it looks like this is the only issue (currently tracked in
http://bugs.python.org/issue28931).


> I tried to fix urllib so that it's guaranteed to call voidresp(), and
> failed. Too complicated, effectively reimplementing a sizable part of FTP
> client logic is required, perhaps even monkey-patching ftplib to be able to
> set flags in the right places.
>

Why did you fail? Why urllib can't close() -> voidresp() the FTP session
once it's done with it?


> With simple commands (voidcmd) and self-contained transfer commands
> (retrlines/retrbinary), ftplib does incapsulate statefulness, by handling
> them atomically. But with transfercmd(), it returns control to the user
> halfway through.
>

That's by design. ftplib's transfercmd() just works like that: it's a low
level method returning a "data" socket and it's just up to you to cleanly
close the session (close() -> voidresp()) once you're done with it. Most of
the times you don't even want to use transfercmd() in the first place, as
you just use stor* and retr* methods.


> At this point, if ftplib doesn't itself keep track that the control
> connection is currently in invalid state for the next command, the user is
> forced to do that themselves instead.
>

Hence why I suggested a docfix.


> And guess what - urllib has to use ftplib through a wrapper class that
> does exactly that. That's definitely a "stock logic part" 'cuz it's an
> integral part of FTP rather than something specific to user logic.
>

That wrapper should just cleanly close the session.


> The other "stock logic part" currently missing is error handling during
> transfer. If the error's cause is local (like a local exception that
> results in closing the data socket prematurely, or the user closing it by
> hand, or an ABOR they sent midway), the error code from the end-of-transfer
> response should be ignored, otherwise, it shouldn't.
>

Absolutely not. Base ftplib should NOT take any deliberate decision such as
ignoring an error.
I can even come up with use cases: use ftplib to test FTP servers so that
they *do* reply with an error code in certain conditions. Here's a couple
examples in pyftpdlib:

https://github.com/giampaolo/pyftpdlib/blob/17bebe3714752b01e43ab60d2771308f4594bd99/pyftpdlib/test/test_functional.py#L1354-L1369

https://github.com/giampaolo/pyftpdlib/blob/17bebe3714752b01e43ab60d2771308f4594bd99/pyftpdlib/test/test_functional.py#L1973-L1980


> Other than making ftplib keep track of the session's internal state while
> the user has control and providing the custom error handling logic to them,
> another solution is to never release control to the user midway, i.e. ban
> transfercmd() entirely and only provide self-contained
> retr*()/dir()/whatever, but let the callback signal them to abort the
> transfer. That way, the state would be kept implicitly - in the execution
> pointer.
>

Banning transfercmd() means renaming it to _transfercmd() which is a no-no
for backward compatibility. Also, as I've shown above, transfercmd() does
have a use case which relies on it behaving like that (return control
midway).


-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Document C API that is not part of the limited API

2016-12-28 Thread Brett Cannon 
On Tue, 27 Dec 2016 at 12:15 Ronald Oussoren  wrote:

>
> > On 27 Dec 2016, at 20:04, Serhiy Storchaka  wrote:
> >
> > From the documentation:
> >
> > https://docs.python.org/3/c-api/stable.html
> >
> >In the C API documentation, API elements that are not part of the
> limited API are marked as "Not part of the limited API."
> >
> > But they don't.
> >
> > I prepared a sample patch that adds the notes to Unicode Objects and
> Codecs C API (http://bugs.python.org/issue29086). I'm going to add them
> to all C API.
> >
> > What are your though about formatting the note? Should it be before the
> description, after the description, but before the "deprecated" directive
> (as in the patch), or after the first paragraph of the description? Should
> it be on the separate line or be appended at the end of the previous
> paragraph, or starts the first paragraph (if before the description)? May
> be introduce a special directive for it?
>
> A directive would make it easier to ensure that the text about the stable
> API is consistent.  I’d also consider adding that directive to all API’s
> that *are* part of the stable API instead of the other way around (that
> would also require changes to …/stable.html). That would have two
> advantages: firstly it makes it easier to document from which version an
> API is part of the stable ABI, and secondly forgetting the annotation would
> imply that an API is not part of the stable ABI instead of accidentally
> claiming to increase the stable ABI.
>

I like Ronald's suggestion of both using a directive and making it for the
stable ABI since it should be an opt-in thing for the API to be stable
instead of opt-out.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Document C API that is not part of the limited API

2016-12-28 Thread Steve Dower 

On 28Dec2016 1145, Brett Cannon wrote:

On Tue, 27 Dec 2016 at 12:15 Ronald Oussoren mailto:[email protected]>> wrote:
A directive would make it easier to ensure that the text about the
stable API is consistent.  I’d also consider adding that directive
to all API’s that *are* part of the stable API instead of the other
way around (that would also require changes to …/stable.html). That
would have two advantages: firstly it makes it easier to document
from which version an API is part of the stable ABI, and secondly
forgetting the annotation would imply that an API is not part of the
stable ABI instead of accidentally claiming to increase the stable ABI.


I like Ronald's suggestion of both using a directive and making it for
the stable ABI since it should be an opt-in thing for the API to be
stable instead of opt-out.


The directive is a good idea, but I'm a little concerned about the 
stable API being opt-out in the headers and opt-in in the documentation.


Perhaps we should also figure out the preprocessor gymnastics we need to 
make it opt-in in the headers too? Though once we get the build 
validation to detect when the stable API changes accidentally it'll be 
less of an issue.


Cheers,
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 514 and pywin32

2016-12-28 Thread Glenn Linderman 

Apparently the gohlke site had done the wheel convert, as they had a wheel.
The post-install steps were needed to make things work for me, and were 
documented at the gohlke site.


Thanks to both of you for your help.

On 12/28/2016 9:09 AM, Steve Dower wrote:
It's likely that they're using the broken version of bdist_wininst for 
their installer (I thought Mark reported the issue and had a 
workaround though...). It's already fixed, but hasn't been released yet.


Another workaround is to use "wheel convert" on the exe and then 
install the wheel. You miss out on their post-install steps, but most 
people don't need those anyway.


Cheers,
Steve

Top-posted from my Windows Phone

From: Burkhard Meier 
Sent: ‎12/‎28/‎2016 1:43
To: Glenn Linderman 
Cc: Python Dev 
Subject: Re: [Python-Dev] PEP 514 and pywin32

Try the 'gohlke' download site. Whenever getting stuck in some of the 
newest Python 3.x versions, that side usually has installers that 
work. It did work for me just now, using Python 3.6.0 64-bit on 
Windows 10 64-bit OS.


Inline image 1

Burkhard

On Tue, Dec 27, 2016 at 9:17 PM, Glenn Linderman > wrote:


So today I tried to install pywin32 on my new Python 3.6.0 and got
the following error:

---
Cannot install
---
Python version 3.6-32 required, which was not found in the registry.
---
OK
---

Seems like pywin32, although built for 3.6, doesn't understand or
conform to the PEP 514? So the installer doesn't work? I suspect
maybe the code would still work, if it would install. I also noted
that pip cannot find a compatible pywin32, and PyPI only reports
compatibility through Python 3.3.

1. Where should this be reported? SourceForge?
2. Anyone know a workaround?

-- 
Glenn


Experience is that marvelous thing that enables you to recognize a
mistake when you make it again. -- Franklin Jones

___
Python-Dev mailing list
[email protected] 
https://mail.python.org/mailman/listinfo/python-dev

Unsubscribe:

https://mail.python.org/mailman/options/python-dev/burkhardameier%40gmail.com






___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/v%2Bpython%40g.nevcal.com


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for feedback about fixing `ftplib' (issue25458)

2016-12-28 Thread Ivan Pozdeev via Python-Dev 
Re: [Python-Dev] Asking for feedback about fixing `ftplib' (issue25458) 
On 28.12.2016 21:02, Giampaolo Rodola' wrote:



On Wed, Dec 28, 2016 at 3:57 PM, Ivan Pozdeev > wrote:


On 25.12.2016 17:21, Giampaolo Rodola' wrote:

From what I understand the problem should be fixed in urllib so
that it always closes the FTP connection. I understand this is
what happens in recent 3.x versions but not on 2.7.

urllib in 2.7 should indeed be fixed to close FTP connections
rather than leave them dangling, no question about that.


To me it looks like this is the only issue (currently tracked in 
http://bugs.python.org/issue28931).
So, closing the control connection ungracefully and without checking the 
response is a non-issue? The right way to do it?


I tried to fix urllib so that it's guaranteed to call voidresp(),
and failed. Too complicated, effectively reimplementing a sizable
part of FTP client logic is required, perhaps even monkey-patching
ftplib to be able to set flags in the right places.


Why did you fail? Why urllib can't close() -> voidresp() the FTP 
session once it's done with it?
A simple voidresp() in a socket close handler doesn't work here. That's 
why I failed.
* Sometimes, the error needs to be ignored, and sometimes, it doesn't. 
With the current architecture, I cannot check which is the case.
* Error checking can't be done in a close handler. Close handlers 
(including this one) are called in finally blocks, raising an exception 
here would disrupt further cleanup actions, leaving objects in an 
undefined state, and mask the current exception if there's any.


What is more important, the entire urllib.ftpwrapper rubs me the wrong way.
urllib's use case is pretty standard. All it does is the basic FTP 
premise of log in, retrieve, log out. So, logically speaking, it 
shouldn't have to implement loads upon loads of custom logic on top of 
ftplib if ftplib is to be called a production-ready library. Any logic 
it has to implement shall be strictly application-specific.


Since it has to, there can only be two possible conclusions: either it 
uses it not in the way intended, or ftplib is an incomplete library and 
is missing some critical parts that are required for practical use.



I won't comment on anything further on that boils down to these two points.
As long as I fail to bring them across, any further discussion is moot.


With simple commands (|voidcmd|) and self-contained transfer
commands (retrlines/retrbinary), ftplib does incapsulate
statefulness, by handling them atomically. But with transfercmd(),
it returns control to the user halfway through.


That's by design. ftplib's transfercmd() just works like that: it's a 
low level method returning a "data" socket and it's just up to you to 
cleanly close the session (close() -> voidresp()) once you're done 
with it. Most of the times you don't even want to use transfercmd() in 
the first place, as you just use stor* and retr* methods.


At this point, if ftplib doesn't itself keep track that the
control connection is currently in invalid state for the next
command, the user is forced to do that themselves instead.


Hence why I suggested a docfix.

And guess what - urllib has to use ftplib through a wrapper class
that does exactly that. That's definitely a "stock logic part"
'cuz it's an integral part of FTP rather than something specific
to user logic.


That wrapper should just cleanly close the session.

The other "stock logic part" currently missing is error handling
during transfer. If the error's cause is local (like a local
exception that results in closing the data socket prematurely, or
the user closing it by hand, or an ABOR they sent midway), the
error code from the end-of-transfer response should be ignored,
otherwise, it shouldn't.


Absolutely not. Base ftplib should NOT take any deliberate decision 
such as ignoring an error.
I can even come up with use cases: use ftplib to test FTP servers so 
that they *do* reply with an error code in certain conditions. Here's 
a couple examples in pyftpdlib:


https://github.com/giampaolo/pyftpdlib/blob/17bebe3714752b01e43ab60d2771308f4594bd99/pyftpdlib/test/test_functional.py#L1354-L1369

https://github.com/giampaolo/pyftpdlib/blob/17bebe3714752b01e43ab60d2771308f4594bd99/pyftpdlib/test/test_functional.py#L1973-L1980

If you claim that testing a server is an intended use case for ftplib 
(rather than meddling in its internals at your own risk), you need to 
document all the other low-level functions that you use (like putcmd) 
for your claim to stand. The currently published interface doesn't offer 
a sufficiently fine-grained control for that use case. Its 
line-of-support is "elementary compounds", none of which leaves the 
control connection in an invalid state.


Other than making ftplib keep track of the session's internal
state while the