[issue16038] ftplib: unlimited readline() from connection

[Jeff Dafoe]

Jeff Dafoe  added the comment:

I have a question about this old patch, as it just came down in a CentOS 6 
update. I think the patch is applied to the data channel in ASCII mode and not 
just the control channel. On the data channel in ASCII mode, there should be no 
assumption of maximum line length before EOL. I saw that your current value 
came from vsftpd's header file. I'm guessing if you look at the implementation, 
it's either only applied to the control channel or it's just used to set a 
single read size inside of a loop.  Examples of ASCII mode files that can 
exceed nearly any MAXLINE value without EOL are XML files or EDI files.

--
nosy: +Jeff Dafoe

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Georg Brandl]

Changes by Georg Brandl ge...@python.org:


--
versions:  -Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Roundup Robot]

Roundup Robot added the comment:

New changeset 783e7b4375ac by Georg Brandl in branch '3.2':
Issue #16038: CVE-2013-1752: ftplib: Limit amount of data read by
https://hg.python.org/cpython/rev/783e7b4375ac

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Georg Brandl]

Changes by Georg Brandl ge...@python.org:


--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Berker Peksag]

Changes by Berker Peksag berker.pek...@gmail.com:


--
stage: patch review - resolved

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Radu Voicilas]

Radu Voicilas added the comment:

I'm a little confused about this patch. Please correct me if I'm wrong, but 
fp.readline([size + 1]) should return a line of length at most size + 1. This 
means that the check len(line)  size will always be true when reading a line 
that has a length greater than self.maxline. Also, wouldn't it make more sense 
to have the line that logs stuff in debugging mode be before raising a 
LineTooLong exception ? This way you have the option of actually seeing the 
line.

--
nosy: +raduv

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[A.M. Kuchling]

A.M. Kuchling added the comment:

Are we likely to actually apply this change to the 3.1 and 3.2 branches, given 
that even the later 3.3 branch is now in security-fix mode?  If we're not going 
to change 3.1 or 3.2, this issue can just be closed.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Roundup Robot]

Roundup Robot added the comment:

New changeset 44ac81e6d584 by Serhiy Storchaka in branch '2.7':
Issue #16038: CVE-2013-1752: ftplib: Limit amount of data read by
http://hg.python.org/cpython/rev/44ac81e6d584

New changeset 38db4d0726bd by Serhiy Storchaka in branch '3.3':
Issue #16038: CVE-2013-1752: ftplib: Limit amount of data read by
http://hg.python.org/cpython/rev/38db4d0726bd

New changeset 0c48fe975c54 by Serhiy Storchaka in branch 'default':
Issue #16038: CVE-2013-1752: ftplib: Limit amount of data read by
http://hg.python.org/cpython/rev/0c48fe975c54

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
versions:  -Python 2.7, Python 3.1, Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Arfrever Frehtes Taifersar Arahesis]

Arfrever Frehtes Taifersar Arahesis added the comment:

(3.1 branch is open to security fixes.)

--
versions: +Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

You are right. I will try to provide patches for other Python versions
later next week.

On Sun, Oct 20, 2013 at 5:08 PM, Arfrever Frehtes Taifersar Arahesis 
rep...@bugs.python.org wrote:


 Arfrever Frehtes Taifersar Arahesis added the comment:

 (3.1 branch is open to security fixes.)

 --
 versions: +Python 3.1

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue16038
 ___


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

I think this is already fixed. Barry can we close this out?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Arfrever Frehtes Taifersar Arahesis]

Arfrever Frehtes Taifersar Arahesis added the comment:

It is fixed in Python 2.6, but not 2.7, 3.1, 3.2, 3.3, 3.4.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Larry Hastings]

Larry Hastings added the comment:

Ping.  Please fix before beta 1.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Please apply it yourself.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Roundup Robot]

Roundup Robot added the comment:

New changeset 8b19e7d0be45 by Barry Warsaw in branch '2.6':
- Issue #16038: CVE-2013-1752: ftplib: Limit amount of data read by
http://hg.python.org/cpython/rev/8b19e7d0be45

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Changes by Barry A. Warsaw ba...@python.org:


--
versions:  -Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

I believe the problem is the set of next_retr_data attribute here:

def test_retrlines_too_long(self):
self.server.handler.next_retr_data = 'x' * self.client.maxline * 2

...because self.server.handler runs in a different thread (different than the 
main one, which is where the setattr() occurs).
We should introduce a new command in the dummy FTP server which sets 
next_retr_data from within the server thread itself. Will try to work on a 
patch later this week (I'm sorry but I can't make it earlier).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

On Sep 24, 2013, at 01:12 PM, Giampaolo Rodola' wrote:

Giampaolo Rodola' added the comment:

I believe the problem is the set of next_retr_data attribute here:

def test_retrlines_too_long(self):
self.server.handler.next_retr_data = 'x' * self.client.maxline * 2

...because self.server.handler runs in a different thread (different than the
main one, which is where the setattr() occurs).  We should introduce a new
command in the dummy FTP server which sets next_retr_data from within the
server thread itself. Will try to work on a patch later this week (I'm sorry
but I can't make it earlier).

+1 - that explanation makes a lot of sense, thanks!

Currently 2.6.9rc1 is planned for Monday 30-September.  It would be nice to
get this one in before then, but if not that's okay.  I think it's fairly low
risk.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch.

--
Added file: http://bugs.python.org/file31862/ftplib_maxline.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

On Sep 24, 2013, at 09:59 PM, Serhiy Storchaka wrote:

Added file: http://bugs.python.org/file31862/ftplib_maxline.patch

This looks great and fixes the test failure problem.  Thanks!  Serhiy, please
feel free to apply this to the 2.6 branch, or let me know if you'd rather I
apply it.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What about time.sleep(0.1)?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
nosy: +josiahcarlson, stutzbach

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

On Sep 23, 2013, at 03:36 PM, Serhiy Storchaka wrote:

What about time.sleep(0.1)?

I usually don't like introducing sleeps to fix race conditions, but if that's
the only option for landing this patch, maybe we'll have to hold our noses and
do it.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

Barry can you paste the traceback caused by the race condition? What's not 
clear to me is when (what line) it occurs.
One solution might be to send a NOOP command (self.client.sendcmd('noop')) in 
order to synchronize client and server.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

On Sep 23, 2013, at 06:33 PM, Giampaolo Rodola' wrote:

Barry can you paste the traceback caused by the race condition? What's not
clear to me is when (what line) it occurs.  One solution might be to send a
NOOP command (self.client.sendcmd('noop')) in order to synchronize client
and server.

There's no traceback other than the test failure that I posted.  It's a race
condition because with a little sleep, the test passes.  Without it, it fails.

This is on various flavors of Ubuntu (only up to 10.04 which is the last
version I can build a full 2.6 against) and Debian.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

Okay, this one is quite odd.  It's definitely a timing issue.

If I put a `import time; time.sleep(1)` at the beginning of 
test_retrlines_too_line() -- i.e. first line of the method -- then the test 
reliably passes.  If I put a `print(len(line))` just before the maxline test in 
FTP.retrlines(), then the test will pass just as reliably.

If I put that retrlines() print *after* the maxline test, then it passes 
sometimes and fails sometimes.  When if fails, it's only ready 12 bytes from 
the `fp.readline()` call.  When it passes, it's reading 8193 bytes (thus 
triggering the expected exception).

I really hate to put a sleep in the test to make it pass.  Obviously it would 
be better not to fudge this race condition, but I don't know the code well 
enough to know where the race is yet.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

Looks legitimate to me. I will come up with a separate patch for later Python 
versions.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

Yep, confirmed that ftplib.patch causes test_ftplib to fail, at least on Ubuntu 
10.04 chroot.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

Succeeds on OS X 10.8 (although there are other failures)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[A.M. Kuchling]

A.M. Kuchling added the comment:

For 2.6 I'll make a revised version of Giampaolo's patch that doesn't add a new 
exception class.  

Rationale: Adding a new exception class changes the API of the module, which 
we'd like to avoid.  If someone is writing 2.6 code that wants to catch this 
exception, they can't write except ftplib.LineTooLong because the name isn't 
present.  Instead they'll have to catch the parent Error exception class and 
analyze either its type or the exception message.  My conclusion is that adding 
the new class isn't actually useful.

(bwarsaw and I are at a mini-sprint looking at the 2.6.9 blockers, so we're 
looking at all of these 'unlimited readline' issues and will continue to remove 
new exceptions introduced by patches.)

--
nosy: +akuchling

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[A.M. Kuchling]

A.M. Kuchling added the comment:

2.6 version of the patch.  Changes from Giampaolo's version of the patch:

* 2.6 didn't have FTP over TLS, so the patch changes slightly to adapt.

* Removed the LineTooLong exception class and just raise Error instead.  (This 
repeats the message text for Line too long in several place.)

--
Added file: http://bugs.python.org/file31777/ftplib.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
versions: +Python 2.6, Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

==
FAIL: test_retrlines_too_long (__main__.TestFTPClass)
--
Traceback (most recent call last):
  File Lib/test/test_ftplib.py, line 374, in test_retrlines_too_long
self.client.retrlines, 'retr', received.append)
AssertionError: Error not raised

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

I'm attaching a slightly different patch including new tests and which uses a 
'maxline' class attribute (as opposed to a global var).
Christian if that's OK with you I will wait a while and then make a commit for 
all Python versions.

--
Added file: http://bugs.python.org/file31589/ftplib_maxline.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

blocker for 2.6.9

--
nosy: +barry
priority: critical - release blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Error message got more than %d bytes is misleading because in most cases 
(except storlines()) we read not bytes but a text string.

There are 4 changes in the ftplib module but only one of them covered by test.

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Benjamin Peterson]

Benjamin Peterson added the comment:

I suppose this is fine for 2.7

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Christian Heimes]

Christian Heimes added the comment:

The patches are languishing in the bug tracker for a while...

Benjamin:
I like to apply them to 3.3 and default before the next release of 3.3. Do you 
want to have the fixes in 2.7, too?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Charles-François Natali]

Charles-François Natali added the comment:

So, what now?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Benjamin Peterson]

Benjamin Peterson added the comment:

Not blocking 2.7.4 as discussed on mailing list.

--
priority: release blocker - critical

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

Patch looks ok. Just add the new exception class to all_errors list.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Michał Jastrzębski]

Michał Jastrzębski added the comment:

Thank you Giampaolo,

I'm attaching patch changed according to your suggestion.

--
Added file: http://bugs.python.org/file29090/ftplib_maxline.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Christian Heimes]

Christian Heimes added the comment:

CVE-2013-1752  Unbound readline() DoS vulnerabilities in Python stdlib

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Michał Jastrzębski]

Michał Jastrzębski added the comment:

Hello,

I've set up maxline limit to 8192. Also I've add some changes Antoine suggested 
earlier.

--
Added file: http://bugs.python.org/file29019/ftplib_maxline.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +larry
stage: needs patch - patch review
versions: +Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Antoine Pitrou]

Antoine Pitrou added the comment:

Not sure how I nosied Larry by updating this issue, sorry for the mistake.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ah, but that's because I added 3.4 in the versions field and the issue is a 
release blocker :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Larry Hastings]

Larry Hastings added the comment:

My spies are everywhere!  You cannot hide your black heart, Pitrou.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

LineTooLong should be added to ftplib.all_errors.
4096 looks enough to me.
The longest lines I can think of occur when processing MLSD command which 
produces an output of like this:

type=file;size=156;perm=r;modify=20071029155301;unique=801cd2; music.mp3
type=dir;size=0;perm=el;modify=20071127230206;unique=801e33; ebooks
type=file;size=211;perm=r;modify=20071103093626;unique=801e32; module.py

Considering that the file names listed in there are forced to consist of base 
names (as opposed to *full* path names) I doubt we'll ever hit 4096.
In pyftpdlib I used 2048 bytes.
I can't recall any reference about this in any FTP-related RFC.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Christian Heimes]

Christian Heimes added the comment:

I suggest that we use twice the size of the largest limit (8192) for the DoS 
fix and reduce it to 2048 for Python 3.4. 8192 is still a small number for 
modern computers.

I also like to see comments next to the limit that explain on what grounds we 
have chosen the value. For example

# vfstpd has a limit of 4096 
(ftp://vsftpd.beasts.org/users/cevans/untar/vsftpd-3.0.2/defs.h)
# pyftpdlib has a limit of 2048

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Antoine Pitrou]

Antoine Pitrou added the comment:

 I suggest that we use twice the size of the largest limit (8192) for
 the DoS fix and reduce it to 2048 for Python 3.4. 8192 is still a
 small number for modern computers.

Why do you want to reduce it? It doesn't bring any additional security.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Christian Heimes]

Christian Heimes added the comment:

Thank you very much!

Have you read and checked what the RFCs about the FTP protocol say about  
maximum line length?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Michał Jastrzębski]

Michał Jastrzębski added the comment:

Well its my understanding, that there is no maximum length specified in RFC959. 
There is however option to set it up while telnet connection, and that would be 
other solution to this issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Antoine Pitrou]

Antoine Pitrou added the comment:

Michał, thanks for the patch. Could you sign and e-mail a contributor's 
agreement? http://www.python.org/psf/contrib/

As for the patch:
- the test could be a separate test_ method
- the offset variable isn't used in cmd_retrlarge, there is no need computing it

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Charles-François Natali]

Charles-François Natali added the comment:

 Have you read and checked what the RFCs about the
 FTP protocol say about  maximum line length?

vsftpd seems to use a 4096 limit (and the guy knows his stuff :-):
ftp://vsftpd.beasts.org/users/cevans/untar/vsftpd-3.0.2/defs.h

--
nosy: +neologix

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Michał Jastrzębski]

Michał Jastrzębski added the comment:

Well, that is not from RFC (or I hadn't find it):) however I'd lie if I'd call 
myself an expert, should I change limit to 4096 then?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Charles-François Natali]

Charles-François Natali added the comment:

 Well, that is not from RFC (or I hadn't find it):) however I'd lie if I'd 
 call myself an expert, should I change limit to 4096 then?

It's probably not in the RFC: this just shows that the limit chosen
should be enough.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Michał Jastrzębski]

Michał Jastrzębski added the comment:

Hello,

I've made patch which address this issue.

--
keywords: +patch
nosy: +inc0
Added file: http://bugs.python.org/file28970/ftplib_maxline.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Christian Heimes]

Changes by Christian Heimes li...@cheimes.de:


--
nosy: +benjamin.peterson, georg.brandl
priority: critical - release blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Giampaolo Rodola']

Changes by Giampaolo Rodola' g.rod...@gmail.com:


--
nosy: +giampaolo.rodola

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Christian Heimes]

Changes by Christian Heimes li...@cheimes.de:


--
assignee:  - christian.heimes
priority: normal - critical
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16038] ftplib: unlimited readline() from connection

[Christian Heimes]

New submission from Christian Heimes:

This bug is similar to #16037.

The ftplib module doesn't limit the amount of read data in its call to 
readline(). An erroneous or malicious FTP server can trick the ftplib module to 
consume large amounts of memory.

Suggestion:
The ftplib module should be modified to use limited readline() with _MAXLINE 
like the httplib module.

--
components: Library (Lib)
messages: 171241
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: ftplib: unlimited readline() from connection
type: resource usage
versions: Python 2.7, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com