On Sat, Mar 13, 2010 at 9:24 AM, gregory dudek <du...@cim.mcgill.ca> wrote: > The Telnet module telnetlib.py can be > very slow -- unusably slow -- for large automated data transfers. There are > typically done in raw mode. > > The attached patch greatly increased the speed of telnet interactions in raw > mode. I submitted this a couple of year ago, but it was for an older branch > of python.
For which Python version is your new patch? > There are 2 key things being done: > 1) concatenations string with string.join instead of '+' (which is probably > a minor issue) This looks like superstition in the patch below -- there is no reason why "".join((a, b)) should be any faster than a+b. (The join trick is important when joining *many* strings, but for two it doesn't make a difference, both strings have to be copied at least once no matter what you try.) > 2) wholesale appending the raw and processed buffers when the IAC character > is not found. The should be examined > carefully since I am not an expert in the Telnet protocol, but it seems > to work very well giving me a 5x speedup. This makes sense and is likely the reason your patch is faster. You should really submit this to the bug tracker at bugs.python.org. > --- installed/telnetlib.py 2010-02-02 22:57:58.000000000 -0500 > +++ telnetlib.py 2010-03-13 12:17:02.000000000 -0500 > @@ -30,6 +30,7 @@ > - timeout should be intrinsic to the connection object instead of an > option on one of the read calls only > > +Modified by G. Dudek for greater efficiency. You're gonna need to submit a Python contributor agreement (http://www.python.org/psf/contrib/contrib-form/) if/when your patch gets accepted. Welcome to the club! > """ > > > @@ -420,6 +421,14 @@ > """ > buf = ['', ''] > try: > + if self.rawq: > + if not IAC in self.rawq: > + # speed hack, no IAC just grab whole queue. --Dudek > + buf[self.sb] = "".join((buf[self.sb] , self.rawq)) > + self.cookedq = "".join((self.cookedq , buf[0] )) > + self.sbdataq = "".join((self.sbdataq , buf[1] )) > + self.rawq_flush() > + return > while self.rawq: > c = self.rawq_getchar() > if not self.iacseq: > @@ -428,7 +437,7 @@ > if c == "\021": > continue > if c != IAC: > - buf[self.sb] = buf[self.sb] + c > + buf[self.sb] = "".join((buf[self.sb] , c)) > continue > else: > self.iacseq += c > @@ -480,8 +489,14 @@ > self.iacseq = '' # Reset on EOF > self.sb = 0 > pass > - self.cookedq = self.cookedq + buf[0] > - self.sbdataq = self.sbdataq + buf[1] > + self.cookedq = "".join((self.cookedq , buf[0] )) > + self.sbdataq = "".join((self.sbdataq , buf[1] )) > + > + > + def rawq_flush(self): > + """ Set the queue to empty status """ > + self.rawq = '' > + self.irawq = 0 > > def rawq_getchar(self): > """Get next char from raw queue. > @@ -516,7 +531,7 @@ > buf = self.sock.recv(50) > self.msg("recv %r", buf) > self.eof = (not buf) > - self.rawq = self.rawq + buf > + self.rawq = "".join((self.rawq,buf)) > > def sock_avail(self): > """Test whether data is available on the socket.""" > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com