Re: Python garbage collector/memory manager behaving strangely

2012-11-15 Thread Thomas Rachel
Am 17.09.2012 04:28 schrieb Jadhav, Alok: Thanks Dave for clean explanation. I clearly understand what is going on now. I still need some suggestions from you on this. There are 2 reasons why I was using self.rawfile.read().split('|\n') instead of self.rawfile.readlines() - As you have seen,

Re: Python garbage collector/memory manager behaving strangely

2012-11-14 Thread Aahz
In article 50570de3$0$29981$c3e8da3$54964...@news.astraweb.com, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Mon, 17 Sep 2012 06:46:55 -0400, Dave Angel wrote: On 09/16/2012 11:25 PM, alex23 wrote: def readlines(f): lines = [] while f is not empty:

Re: Python garbage collector/memory manager behaving strangely

2012-11-14 Thread Dieter Maurer
a...@pythoncraft.com (Aahz) writes: ... def readlines(f): lines = [] while f is not empty: line = f.readline() if not line: break if len(line) 2 and line[-2:] == '|\n': lines.append(line) yield

Re: Python garbage collector/memory manager behaving strangely

2012-10-31 Thread Robert Miles
On 9/16/2012 9:12 PM, Dave Angel wrote: On 09/16/2012 09:07 PM, Jadhav, Alok wrote: Hi Everyone, I have a simple program which reads a large file containing few million rows, parses each row (`numpy array`) and converts into an array of doubles (`python array`) and later writes into an `hdf5

Re: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Dave Angel
On 09/16/2012 11:25 PM, alex23 wrote: On Sep 17, 12:32 pm, Jadhav, Alok alok.jad...@credit-suisse.com wrote: - As you have seen, the line separator is not '\n' but its '|\n'. Sometimes the data itself has '\n' characters in the middle of the line and only way to find true end of the line is

RE: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Jadhav, Alok
garbage collector/memory manager behaving strangely On 09/16/2012 11:25 PM, alex23 wrote: On Sep 17, 12:32 pm, Jadhav, Alok alok.jad...@credit-suisse.com wrote: - As you have seen, the line separator is not '\n' but its '|\n'. Sometimes the data itself has '\n' characters in the middle

Re: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Steven D'Aprano
On Mon, 17 Sep 2012 06:46:55 -0400, Dave Angel wrote: On 09/16/2012 11:25 PM, alex23 wrote: def readlines(f): lines = [] while f is not empty: line = f.readline() if not line: break if len(line) 2 and line[-2:] == '|\n':

Re: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Dave Angel
On 09/17/2012 07:47 AM, Steven D'Aprano wrote: On Mon, 17 Sep 2012 06:46:55 -0400, Dave Angel wrote: On 09/16/2012 11:25 PM, alex23 wrote: def readlines(f): lines = [] while f is not empty: line = f.readline() if not line: break if

Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok
Hi Everyone, I have a simple program which reads a large file containing few million rows, parses each row (`numpy array`) and converts into an array of doubles (`python array`) and later writes into an `hdf5 file`. I repeat this loop for multiple days. After reading each file, i delete all

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Dave Angel
On 09/16/2012 09:07 PM, Jadhav, Alok wrote: Hi Everyone, I have a simple program which reads a large file containing few million rows, parses each row (`numpy array`) and converts into an array of doubles (`python array`) and later writes into an `hdf5 file`. I repeat this loop for

RE: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok
, Alok -Original Message- From: Dave Angel [mailto:d...@davea.name] Sent: Monday, September 17, 2012 10:13 AM To: Jadhav, Alok Cc: python-list@python.org Subject: Re: Python garbage collector/memory manager behaving strangely On 09/16/2012 09:07 PM, Jadhav, Alok wrote: Hi Everyone

RE: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok
Angel [mailto:d...@davea.name] Sent: Monday, September 17, 2012 10:13 AM To: Jadhav, Alok Cc: python-list@python.org Subject: Re: Python garbage collector/memory manager behaving strangely On 09/16/2012 09:07 PM, Jadhav, Alok wrote: Hi Everyone, I have a simple program which reads a large

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread alex23
On Sep 17, 12:32 pm, Jadhav, Alok alok.jad...@credit-suisse.com wrote: - As you have seen, the line separator is not '\n' but its '|\n'. Sometimes the data itself has '\n' characters in the middle of the line and only way to find true end of the line is that previous character should be a bar

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread 88888 Dihedral
alex23於 2012年9月17日星期一UTC+8上午11時25分06秒寫道: On Sep 17, 12:32 pm, Jadhav, Alok alok.jad...@credit-suisse.com wrote: - As you have seen, the line separator is not '\n' but its '|\n'. Sometimes the data itself has '\n' characters in the middle of the line and only way to find true end of