[issue23906] poplib maxline behaviour may be wrong

[R. David Murray]

R. David Murray added the comment:

If maxline is too small, messages won't get through.  If maxline is too large 
*huge* messages will get through...and the DDOS danger of exhausting the 
server's resources will occur.  So, we really ought to provide a way to limit 
the maximum message size *anyway*...at which point a separate maxline value 
doesn't make any sense, since the RFC specifies no maximum line size.

I'm much more comfortable setting a large maximum message size than setting a 
large enough maximum line size to permit that size of message consisting of 
mostly a single line.  Since we aren't going to back out the DDOS fix, we have 
to put the limit *somewhere*.  At least in 3.6 we can make it easy for the 
application to set it.  (Programs using earlier versions will just have to 
monkey-patch, unfortunately...which they have to do right now anyway.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[shiyao.ma]

shiyao.ma added the comment:

Instead of setting a MAXSIZE for the email body, rasing up the MAXLINE might be 
more meaningful.


Consider the case of MAXSIZE, it's essentially the same as MAXLINE. If MAXSIZE 
is relatively small, some messages won't pass through. If the MAXSIZE is 
relatively large, then what's the meaning of setting it?


Thus, it might be more practical to increase the value of MAXLINE so that 99% 
messages can pass through.

--
nosy: +introom

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[R. David Murray]

R. David Murray added the comment:

Note that the max message size solution can be applied to the maintenance 
releases as a fix for this issue by choosing a suitable large default message 
size.  The 'feature' part is just the part exposing the size limit in the 
library API...that part is a feature for 3.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[R. David Murray]

R. David Murray added the comment:

Sorry, I was unclear.  In order to implement maximum message size we have to do 
a bit more to the logic than just use the max message size as the readline 
limit.  But it does seem like the right approach to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[Chris Smowton]

Chris Smowton added the comment:

Created #24706 to describe the unflushed connection problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[Chris Smowton]

Chris Smowton added the comment:

Why wouldn't that fix the problem? The issue is poplib not tolerating server 
behaviour seen in the wild, and if you limit by message size not line length 
you shouldn't see this problem?

(Side note, I'm surprised not to have been emailed when you replied, any idea 
what I'm missing?)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[R. David Murray]

R. David Murray added the comment:

Could you open a separate bug for the recovery problem, please?

Using a maximum message size would not solve this problem, but it would give 
the library user control of when it failed, so it is a good feature request.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[Chris Smowton]

Chris Smowton added the comment:

I found the same problem retrieving mail from my ISP's (unknown) POP3 server. I 
was sent an HTML email as one long 50KB line, which naturally broke everything.

Instead of limiting line length, I suggest you should limit total message body 
size, since that's what you're actually trying to defend against here. You 
could also either use the +OK XXX octets line to set a more conservative limit 
(and fail fast if it announces intent to send more than your limit).

As above the workaround was to insert import poplib; poplib._MAXLINE = 100 
at the top of the 'getmail' script.

A side-note: one message that is broken this way causes all future messages to 
fail because poplib does not flush the connection when bailing due to a 'line 
too long' error. If it isn't prepared to read the rest of the incoming data, it 
*must* hang up the connection and re-login to fetch the next message.

--
nosy: +Chris Smowton

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[R. David Murray]

Changes by R. David Murray :


--
versions: +Python 3.4, Python 3.5, Python 3.6 -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[R. David Murray]

R. David Murray added the comment:

The RFC is in fact not clear on this point.  It is entirely possible to read it 
as saying that each line of a mulitline response is limited to 512 octets.  I 
agree, however, that that is not the most reasonable interpretation.  Instead, 
the line length of RETR message lines should be governed by RFC 5322, which 
specifies a maximum line length of 998 octets.

That, however, means that technically dovecot is still broken, since 2048 is 
quite a bit larger than 998.  In reality, it means that the *internet* is 
broken, in that I presume the root of the problem is that there are mail 
originators out there that are not obeying RFC 5322 (and its 
predecessors...this limit goes back to 821/822).

We use 8192 in smtplib, and that hasn't caused any problems...but then again 
smtplib is originating email, not receiving it.  The IMAP protocol has its own 
problems, quite aside from the length of message body lines, so we ended up 
with a very large MAXLINE there.  It may be that we have no choice except to do 
something similar in poplib.

An interesting question in this context is what smtp servers do. since if 
anyone was going to reject messages with overlong lines, it would be the smtp 
server's job to do it.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[Ingo Ruhnke]

Ingo Ruhnke added the comment:

This also breaks mail retrieval from both gmx.de and gmail.com (two rather 
large and popular mail provider). After setting _MAXLINE 
in/usr/lib/python2.7/poplib.py to some arbitrary higher number mail retrieval 
from both services worked fine again.

This this 2048 does definitively looks badly broken.

--
nosy: +Ingo Ruhnke

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23906] poplib maxline behaviour may be wrong

[Remy Blank]

Changes by Remy Blank :


--
nosy: +rblank

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com